I know that my tests were not as controlled as they should be.

Regards

Simon (www.simonlong.co.uk)

I’ve often had discussions with folks on the optimal concurrent VMotion setting, and had always worked on the basis that “less is more”. By that, I mean that if you have a set amount of bandwidth on your VMotion network, having multiple VMs migrating at once will cause each one to run slower, as they’re competing for bandwidth to transfer memory state. In turn, the longer a VM takes to transfer, the more time it leaves for memory pages to go stale and have to be retransmitted. Thus, the longer a VMotion takes, the more data has to be transmitted over the VMotion network.

This always led me to believe that having VMotions occur in series rather than in parallel would yield the best results.

I guess there are other factors at play here, and I need to go back and re-assess (and test!).

But having said that, i have been using 6 simultanious VMotions on a new 2 Host cluster in which i have regularly put either host into maintenance mode thus VMotioning 20+ VM’s. Whilst this process was running we recieved 0 downtime on any of our systems running in the Cluster.

In step 4 of the VMotion process when the Memory Delta is copied between Hosts, i would agree that ideally this needs to be as quick as possible and if there is less bandwidth avaliable this could cause downtime.
But surely running a memory hungry app e.g. Exchange is just as much of an issue? When a VMotion is performed on a VM using a massive ammount of memory the Memory Delta is going to be a LOT bigger than a VM just idling. So in theroy should take a lot longer to transfer the Delta.

For Example;

Slow VMotion due to 6 simultanious VMotions limiting available bandwidth:
Delta = 100MB Transfered at 100Mbit/s = 8.39s

Slow VMotion due to Large Delta from memory hungry app but with normal available bandwidth:
Delta = 1GB Transfered at 1Gbit/s = 8.59s

As far as i know (correct me if im wrong), running a memory hungry application as yet hasn’t caused downtime on a VM Guest OS whilst in VMotion.

If i get a spare moment i might run a Memory Stress Test app whilst running multiple VMotions and see if the ping drops out. If anyone else has a bit of spare time on their hands…feel free to test it for me

With regards to your questioning of the per-VM VMotion times you are correct, it does seem to take less time for each individual VM. But i think you need to question whether you’d prefer a faster per-VM time or a faster overal process time.

Simon

From what I understand, this is how a VMotion occurs:
1) Pre-migration: set up target host, etc…
2) Copy memory, keep track of changes
3) Pause the VM
4) Copy the memory delta
5) Resume the VM on the target host

Obviously you want step 4 to happen as quickly as possible for minimal downtime. I agree that it takes less time to migrate 6 VMs at once than 2 at a time, but the migration time per-VM increases.

With your example above:
6 VMs at once = 1.54 min migration time for each VM
2 VMs at once = 0.75 min migration time for each VM (2.24 / 3)

So if we assume that step 4 above is typically a percentage of the total migration time, it’s possible you’ve doubled the downtime of the VM during the migration.

Maybe there are syslog messages generated during a VMotion and would be more accurate than a ping to see the exact time a VM was stopped and started.

Just a thought…

PS – This is a good PDF for a technical explanation of VMotion: http://www.vmware.com/pdf/usenix_vmotion.pdf

Simon

Simon