Posts Tagged ‘VMotion’

Uptime lost during VMotion

January 11th, 2009

That’s right. We lose uptime during every VMotion. Relax just a bit – I’m not talking about actual uptime/downtime availability of the guest VM in the datacenter. I’m speaking to the uptime performance metric tracked in the VirtualCenter database. It’s a bug that was introduced in VirtualCenter 2.0 and has remained in the code to this day in VirtualCenter 2.5.0 Update 3. Here’s how it works:

We’ve got a VM named Exchange1. VirtualCenter displays its uptime as 28 days is indicated by the screenshots below:

1-11-2009 12-04-06 AM

1-11-2009 12-10-14 AM

Now the VM has been VMotioned from host SOLO to host LANDO. Notice what has happened to uptime. It has disappeared from the console:

1-11-2009 12-07-37 AM

The real proof in what has ultimately happened is we see from the performance chart the latest uptime metric has been reset from 27.99 days as shown above to 0.0026620 days:

1-11-2009 12-10-54 AM

Sometimes the VIC console will show the uptime counter start over at 0 days, then on to 1 days, etc. Other times the uptime counter will remain blank for days or weeks as you can see from my three other VMs in the first screenshot which show no uptime.

This brings us to an interesting discussion. What would you like uptime in the VIC to mean exactly? Following are my observations and thoughts on VMware’s implementation of the uptime metric in VirtualCenter.

In previous versions of VirtualCenter, a soft reboot of the VM inside of the OS would reset the uptime statistic in VirtualCenter. I believe this was a function of VMware Tools that triggered this.

Today in VirtualCenter 2.5.0 Update 3, a soft reboot inside the guest VM does not reset the uptime statistic back to zero.

A VM which has no VMware Tools installed that is soft rebooted inside of the OS (ie. we’re not talking about any VMware console power operation here) does not reset the uptime statistic.

I could see the community take a few different sides on this as there are two variations of the definition of uptime we’re dealing with here. Uptime of the guest VM OS and uptime of the VM’s virtual hardware.

  1. Should uptime translate into how long the VM virtual hardware has been powered on from a virtual infrastructure standpoint?
  2. Or should uptime translate into how long the OS inside the VM has been up, tracked by VMware Tools?

The VMware administrator cares about the length of time a VM has been powered on. It is the powered on VM that consumes resources from the four resource food groups and impacts capacity.

The guest VM OS administrator, whether it be Windows or Linux, cares about uptime of the guest OS. The owner of the OS is held to SLAs by the business lines.

My personal opinion is that the intended use of the Virtual Infrastructure Client is for the VMware administrator and thus should reflect virtual infrastructure information. My preference is that the uptime statistic in VirtualCenter tracks power operations of the VM irregardless of any reboot sequences of the OS inside the VM. In other words, uptime is not impacted by VMware Tools heartbeats or reboots inside the guest VM. The uptime statistic should only be reset when the VM is powered off or power reset (including instances where HA has recovered a VM).

At any rate, due to the bug that uptime has in VirtualCenter 2.0 and above, it’s a fairly unreliable performance metric for any virtual infrastructure using VMotion and DRS. Furthermore, the term itself can be misleading depending on the administrators interpretation of uptime versus what’s written in the VirtualCenter code.

I submitted a post in VMware’s Product and Feature Suggestions forum in January of 2007 recording the uptime reset on VMotion issue. As this problem periodically bugs me, I followed up a few times. Once in a follow up post in the thread above, and at least one time externally requesting someone from VMware take a look at it. Admittedly I do not have an SR open.

VMware, can we get this bug fixed? After all, if the hypervisor has become an every day commodity item leaving the management tools as the real meat and potatoes, you should make sure our management tools work properly.

Thank you,

Jas

A great disturbance in the Force

December 15th, 2008

Today I felt a great disturbance in the Force, as if millions of voices cried out in terror.  Mohamed Fawzi of the blog Zeros & Ones posted a VMware vs Hyper-V comparison that I felt was neither fair nor truthful.  In fact, I think it is the worst bit of journalism I’ve witnessed in quite a while and even in the face of the VMworld 2008/Microsoft Hyper-V poker chip fiasco, I don’t know if Microsoft would even endorse this tripe.

I didn’t have a lot of time today for rebuttal and thus following are my brief responses:

Cost: It is impossible to summarize cost of a product (and TCO) in one short sentence as you have done.

Support: VMware was the first virtualization company to be listed on the Microsoft SVVP program.  Enough said about that.  If you want to talk about Linux, VMware supports many distros.  Hyper-V last time I checked supports one.

Hardware Requirements: No comparison.  Microsoft does not have VMotion/hot migration or similar.  New server “farms” are not necessarily needed, although a rolling upgrade can be performed using Enhanced VMotion Compatibility where the majority of the technology that will allow this comes from the processor hardware vendors.

Advanced Memory Management: Content based page sharing is a proven technology that I use in a production environment with no performance impacts.  Microsoft does not have this technology and therefore forces their customers to achieve higher consolidation ratios by spending more money on RAM than what would be needed in a VMware datacenter.  Other memory overcommit technologies such as ballooning and swapping come with varying levels of penalty and VMware offers the flexibility to the customer as to what they would like to do in these areas.  Microsoft offers no flexibility or choices.

Hypervisor: ESXi embedded is 32MB.  ESXi installable is about 1GB.  Hyper-V’s comparable products once installed are 1GB and in the 4-10GB neighborhood.  Your point of the Hyper-V hypervisor being 872KB, whether truth or not, bears no relevance for comparison purposes.

Drivers Support: VMware maintains tight control which fosters platform stability.  Installation of XYZ drivers and software adds to instability, support costs, and down time.

Processor Support: False.  ESX/ESXi operates on x86 32bit and x64 64bit processors.  Current 3rd party vendor neutral performance benchmarking between ESX and Hyper-V shows no performance degradation in ESX compared to Hyper-V as a result of address translation or otherwise.  A more truthful headline to be exposed here is Hyper-V isn’t compatible with 32-bit hardware.  Why didn’t you mention this in your Hardware Requirements section?

Application Support: I don’t see any Windows support issues.  Again I remind you, VMware is certified on the Microsoft SVVP program.  Another comparison is made with a particular VMotion restriction.  I’ll grant you that if you admit Microsoft has no VMotion or hot migration at all.

Product Hypervisor Technology: We already covered this in the Drivers Support section.

Epic virtualization and storage blogger Scott Lowe provides his responses here.

Mohamed Fawzi, while it is nice to meet you, it is unfortunate that we met under these terms.  Having just discovered your blog today, I hope you don’t mind if I take a look at some of your other material as it looks like you’ve been at the blogging for a while (much longer than I).  I hope to find some good and interesting reads.

Symantec declares VMware VMotion unsupported

November 18th, 2008

Bad news for VMware VI Enterprise customers everywhere. I just found out I have 110 unsupported production and development VMs in my datacenter. Symantec published Document ID 2008101607465248 on 10/15/08 removing VMware VMotion support from its Symantec Antivirus (SAV) and Symantec Endpoint Protection (SEP) products.

Operating systems impacted are: All Windows operating systems.

Reported issues include but are not limited to:

  • Client communication problems
  • Symantec Endpoint Protection Manager (SEPM) communication issues
  • Content update failures
  • Policy update failures
  • Client data does not get entered into the database
  • Replication failures

This is of grave concern as many enterprise datacenters and VDI deployments are going to be impacted. My personal take is that someone jumped the gun in publishing a document with mysteriously vague detail, but we’ll have to wait and see what shakes out.

I hope that VMware can approach Symantec to get this resolved ASAP. It’s in everyone’s best interest.

Thank you vinternals for the heads up on this.

Update: Symantec has updated their support document stating that the problems a few customers have seen may or may not be related to VMware and VMotion. Until further notice, Symantec is supporting their products on VMware with VMotion. If you experience an issue with Symantec products, please contact Symantec technical support. This confirms my opinion that someone at Symantec jumped the gun by issuing the 10/15/08 support document stating VMware and VMotion is unuspported. Everyone can breathe a sigh of relief now. Or at least I can.

Live migration between CPU vendors demonstrated by AMD and Red Hat

November 11th, 2008

Live migration (VMotion in VMware speak) across AMD and Intel processors is a feature we don’t have today and a technology that many would describe as nearly impossible.

The capability could be in your datacenter sooner than you think. Last Thursday, the Inquirer published an article along with a video where Red Hat and AMD demonstrate the process (of course using streaming video and sound to drive home the point of no interruption) proving that it is possible and the technology to do so may not be so far off. The article goes on to explain that not only can live migration occur between CPU vendors, the same or similar technology can be used to live migrate between CPU architectures from the same vendor (ie. AMD Barcelona Opteron <–> AMD Shanghai Opteron).

Take a look at the video: