boche.net – VMware Virtualization Evangelist

Posts Tagged ‘Documentation’

QuickPress – VMs Per…

May 7th, 2010

I’m trying out my frist QuickPress. Let’s see how this turns out.
Right off the bat, I’m missing the autocomplete feature for Tags. As it turns out, typing more than three lines in the small content box isn’t much fun.

On with the VMware content… This all comes from the VMware vSphere Configuration Maximums document. I’ve bolded some of what I’d call core stats which capacity planners or architects would need to be aware of on a regular basis:

15,000 VMs registered per Linked-mode vCenter Server
10,000 powered on VMs per Linked-mode vCenter Server
4,500 VMs registered per 64-bit vCenter Server
4,000 VMs concurrently scanned by VUM (64-bit)
3,000 powered on VMs per 64-bit vCenter Server
3,000 VMs registered per 32-bit vCenter Server
3,000 VMs connected per Orchestrator
2,000 powered on VMs per 32-bit vCenter Server
1,280 powered on VMs per DRS cluster
320 VMs per host (standalone)
256 VMs per VMFS volume
256 VMs per host in a DRS cluster
200 VMs concurrently scanned by VUM (32-bit)
160 VMs per host in HA cluster with 8 or fewer hosts (vSphere 4.0 Update 1)
145 powered on Linux VMs concurrently scanned per host
145 powered on Linux VMs concurrently scanned per VUM server
145 VMs per host scanned for VMware Tools
145 VMs per host scanned for VMware Tools upgrade
145 VMs per host scanned for virtual machine hardware
145 VMs per host scanned for virtual machine hardware upgrade
145 VMs per VUM server scanned for VMware Tools
145 VMs per VUM server scanned for VMware Tools upgrade
100 VMs per host in HA cluster with 8 or fewer hosts (vSphere 4.0)
72 powered on Windows VMs concurrently scanned per VUM server
40 VMs per host in HA cluster with 9 or more hosts
10 powered off Windows VMs concurrently scanned per VUM server
6 powered on Windows VMs concurrently scanned per host
6 powered off Windows VMs concurrently scanned per host
5 VMs per host concurrently remediated

Got all that?

Update 5/10/10: Added the row 160 VMs per host in HA cluster with 8 or fewer hosts (vSphere 4.0 Update 1) – Thanks for the catch Matt & Joe!

3 comments »

Posted in Virtualization

Tags: Blog Documentation ESX ESXi VMware WordPress

vpxd.cfg Advanced Configuration

March 13th, 2010

vpxd.cfg is an XML formatted file which can be modified to alter the native behavior of the VMware vCenter Server. Sparse references on the internet document the changes that can be made in this environment. Inspired by Ulli Hankeln, the purpose of this blog post is to collect and document all known, unknown, supported, and unsupported vpxd.cfg modifications in a centralized location.

If you have any to add, please provide feedback in the form of a blog comment along with a link pointing to a reference and I’ll update the post.

**Disclaimer**
As with anything found on this site and much of the internet in general, information is provided “as is” without warranty. Modify settings at your own risk. I suggest thoroughly researching the changes first and also checking with VMware Support.

The vpxd.cfg file is located on the VMware vCenter Server by default at %ALLUSERPROFILE%\Application Data\VMware\VMware VirtualCenter\vpxd.cfg

On Windows Server 2008, this would generally be C:\ProgramData\VMware\VMware VirtualCenter\vpxd.cfg
On Windows Server 2003, this would generally be C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\vpxd.cfg

This collection of vpxd.cfg settings has been sourced from various places. The parameters will generally apply to a version of vCenter Server ranging from 2.0 through 4.x. A given parameter can apply to several or even all versions. However, one thing I didn’t do was specify which version of vCenter Server the parameter applies to – too much work – sorry – you’ll have to experiment in your lab or DEV environment. I do think it’s safe to say that most of these parameters focus on the latest releases of vCenter Server – 2.5 and 4.0.

Remember to restart the VMware VirtualCenter Server service in the Server Manager for changes to vpxd.cfg to take effect.

Tag: blockingTimeoutSeconds
Nested In: vmomi, soapStubAdapter
What It Does: Defines the timeout value in seconds for SOAP layer blocking. Use cases for increasing: slow connections, low bandwidth, or high latency between virtual infrastructure components. Read more here and here.
Example:

<vmomi>
<soapStubAdapter>
<blockingTimeoutSeconds>10800</blockingTimeoutSeconds>
</soapStubAdapter>
</vmomi>

Tag: calls
Nested In: trace, vmomi
What It Does: Unknown. Read more here.
Example:

<trace>
<vmomi>
<calls>true</calls>
</vmomi>
</trace>

Tag: cipherList
Nested In: vmacore, ssl
What It Does: Reverts to cipher suites used in previous versions of vCenter Server (2.5u3 and earlier) for browser/SSL compatibility issues. Read more here.
Example:

<vmacore>
<ssl>
<cipherList>DEFAULT</cipherList>
</ssl>
</vmacore>

Tag: compressOnRoll
Nested In: log
What It Does: Defines whether or not vCenter Server vpxd log files are rolled up and compressed into .gz files. Read more here.
Example:

<log>
<compressOnRoll>false</compressOnRoll>
</log>

Tag: cpuFeatureMask
Nested In: guestOSDescriptor, esx-2-x-x, all-versions, all-guests
What It Does: Masks CPU features to force VMotion compatibility between hosts. VMware neither supports nor recommends modifying the VMotion constraints for CPU features. Read more here.
Example:

<guestOSDescriptor>
<esx-2-x-x>
<all-versions>
<all-guests>
<cpuFeatureMask>Elements and mask definition go in here</cpuFeatureMask>
</all-guests>
</all-versions>
</esx-2-x-x>
</guestOSDescriptor>

Tag: directory
Nested In: log
What It Does: Defines the location for the vCenter logs. Read more here.
Example:

<log>
<directory>D:\VC_Logs</directory>
</log>

Tag: dontstartconsolidation
Nested In: vcp2v
What It Does: May resolve an issue where the Consolidation button is missing in the Virtual Infrastructure Client. Read more here.
Example:

<vcp2v>
<dontstartconsolidation>true</dontstartconsolidation>
</vcp2v>

Tag: filterOverheadLimitIssues
Nested In: vpxd
What It Does: Unknown.
Example:

<vpxd>
<filterOverheadLimitIssues>true</filterOverheadLimitIssues>
</vpxd>

Tag: hostRescanFilter
Nested In: unknown
What It Does: Defines the behavior of mass ESX(i) host rescans of vmHBAs. Read more here.
Example:

<hostRescanFilter>true</hostRescanFilter>

Tag: IoMax
Nested In: vmacore, threadpool
What It Does: Unknown but my guess is it defines the maximum I/O for the vpxd.exe process (vCenter Server service). Influenced by TaskMax.
Example:

<vmacore>
<threadpool>
<IoMax>200</IoMax>
</threadpool>
</vmacore>

Tag: level
Nested In: log
What It Does: Defines the logging level for vCenter logs. Read more here.
Example:

<log>
<level>trivia</level>
</log>

Tag: logLevel
Nested In: trace, vmomi
What It Does: Enables debug logging level for vmomi? Read more here.
Example:

<trace>
<vmomi>
<logLevel>verbose</logLevel>
</vmomi>
</trace>

Tag: loglevel
Nested In: nfc
What It Does: Enables debug logging level for the NFC process. Read more here.
Example:

<nfc>
<loglevel>debug</loglevel>
</nfc>

Tag: managedIP
Nested In: unknown
What It Does: Defines the managed IP address used in vCenter Server Heartbeat. Read more here.
Example:

<managedIP>10.10.0.1</managedIP>

Tag: maxCostPerHost
Nested In: ResourceManager
What It Does: Defines the number of simultaneous VM migrations (both hot and cold) per ESX(i) host. Read more here.
Example:

<ResourceManager>
<maxCostPerHost>8</maxCostPerHost>
</ResourceManager>

Tag: maxFileNum
Nested In: log
What It Does: Defines the maximum number of log files for vCenter logs. Read more here.
Example:

<log>
<maxFileNum>50</maxFileNum>
</log>

Tag: maxFileSize
Nested In: log
What It Does: Defines the maximum log file size in Bytes and thus rollover interval for vCenter logs. Read more here.
Example:

<log>
<maxFileSize>10485760</maxFileSize>
</log>

Tag: name
Nested In: log
What It Does: Defines the log file prefix name for vCenter logs. Read more here.
Example:

<log>
<name>vpxd</name>
</log>

Tag: notRespondingTimeout
Nested In: heartbeat
What It Does: Defines the heartbeat timeout value in seconds between ESX(i) hosts and vCenter Server. Use case would be to increase the value if remote ESX(i) hosts frequently go into a not responding state in vCenter Server due to WAN bandwidth or latency issues. Read more here.
Example:

<heartbeat>
<notRespondingTimeout>60</notRespondingTimeout>
</heartbeat>

Tag: portReserveTimeout
Nested In: dvs
What It Does: Defines the timeout value in minutes for unused dvPort reservations. Lowering the value temporarily is helpful for unlocking dvPorts to remove a vDS or dvPort group. Read more here.
Example:

<dvs>
<portReserveTimeout>10</portReserveTimeout>
</dvs>

Tag: serializeadds
Nested In: vpxd, das
What It Does: Unknown but if I had to guess I’d say it defines the behavior of how the HA agent is installed on cluster hosts.
Example:

<vpxd>
<das>
<serializeadds>true</serializeadds>
</das>
</vpxd>

Tag: slotCpuMinMHz
Nested In: vpxd, das
What It Does: Defines the minimum CPU calculation of a HA cluster slot size when there are no CPU reservations. Read more here.
Example:

<vpxd>
<das>
<slotCpuMinMHz>256</slotCpuMinMHz>
</das>
</vpxd>

Tag: slotMemMinMB
Nested In: vpxd, das
What It Does: Defines the minimum memory calculation of a HA cluster slot size when there are no memory reservations. Read more here.
Example:

<vpxd>
<das>
<slotMemMinMB>0</slotMemMinMB>
</das>
</vpxd>

Tag: sspiProtocol
Nested In: unknown
What It Does: Defines the authentication mechanism used with passthrough authentication between the Virtual Infrastructure Client and vCenter Server. Read more here.
Example:

<sspiProtocol>Kerberos</sspiProtocol>

Tag: TaskMax
Nested In: vmacore, threadpool
What It Does: Defines the number of worker threads for the vpxd.exe process (vCenter Server service). Influences IoMax.
Example:

<vmacore>
<threadpool>
<TaskMax>30</TaskMax>
</threadpool>
</vmacore>

Tag: timeout
Nested In: task
What It Does: Defines the timeout value in seconds for long tasks. Read more here.
Example:

<task>
<timeout>10800</timeout>
</task>

Tag: verbose
Nested In: trace, db
What It Does: Enables database tracing. Enables database logging in the vpxd log. Read more here and here.
Example:

<trace>
<db>
<verbose>true</verbose>
</db>
</trace>

Tag: verbosity
Nested In: trace, vmomi
What It Does: Unknown. Read more here.
Example:

<trace>
<vmomi>
<verbosity>verbose</verbosity>
</vmomi>
</trace>

Tag: verboseObjectSize
Nested In: trace, vmomi
What It Does: Unknown. Read more here.
Example:

<trace>
<vmomi>
<verboseObjectSize>40</verboseObjectSize>
</vmomi>
</trace>

Tag: VMOnVirtualIntranet
Nested In: migrate, test, CompatibleNetworks
What It Does: Setting to false enables VMotion for VMs connected to an internal vSwitch. Setting to false will turn off the internal vSwitch restriction on VMotion events. Useful for servers behind a firewall virtual appliance deployed in bridged networking mode. Read more here.
Example:

<migrate>
<test>
<CompatibleNetworks>
<VMOnVirtualIntranet>false</VMOnVirtualIntranet>
</CompatibleNetworks>
</test>
</migrate>

Tag: VMOverheadGrowthLimit
Nested In: cluster
What It Does: Defines the growth rate cap in terms of MB per minute for VM memory overhead at the cluster level. Can be adjusted to resolve high CPU utilization in guest VMs introduced in ESX(i) 3.5 and vCenter 2.5. Read more here.
Example:

<cluster>
<VMOverheadGrowthLimit>5</VMOverheadGrowthLimit>
</cluster>

Slightly related, the vCenter Server process (vpxd.exe) can be launched at a command prompt on the vCenter Server (instead of starting as a service) for troubleshooting purposes. The executable is located at:

<Install Directory>\VMware\Infrastructure\VirtualCenter Server>vpxd.exe

Usage: vpxd.exe [FLAGS]
Flags:
-r Register VMware VirtualCenter Server
-u Unregister VMware VirtualCenter Server
-s Run as a standalone server rather than a Service
-c Print vmdb schema to stdout
-b Recreate database repository
-f cfg Use the specified file instead of the default vpxd.cfg
-l licenseKey Store license key in ldap and assign it to VirtualCenter
-e feature Set the feature to be in use for VirtualCenter. This option takes only one feature at a time.
-p Reset the database password
-v Print the version number to stdout

4 comments »

Posted in Virtualization

Tags: Documentation vCenter Server VMware vSphere

VMkernel Networks, Jumbo Frames, and ESXi 4

February 12th, 2010

Question: Can I implement jumbo frames on ESXi 4 Update 1 VMkernel networks?

Answer: Who in the hell knows?

You see, the ESXi 4.0 Update 1 Configuration Guide states on page 54:

“Jumbo frames are not supported for VMkernel networking interfaces in ESXi.”

Duncan Epping of Yellow Bricks also reports:

“Jumbo frames are not supported for VMkernel networking interfaces in ESXi. (page 54)”

One month after the release of ESXi 4 Update 1, Charu Chaubal of VMware posted on the ESXi Chronicles blog:

“I am happy to say that this is merely an error in the documentation. In fact, ESXi 4.0 DOES support Jumbo Frames on VMkernel networking interfaces. The correction will hopefully appear in a new release of the documentation, but in the meantime, go ahead and configure Jumbo frames for your ESXi 4.0 hosts.”

Shortly after, Duncan Epping of Yellow Bricks confirms Charu Chaubal’s report that jumbo frames are supported on ESXi VMkernel networks.

Now, nearly two months after Charu’s clarification and three months after the release of ESXi 4 Update 1, the documentation remains dubious on page 54 stating that jumbo frames are not supported on ESXi 4 VMkernel networks which is a direct contradition to a VMware ESXi blog.

I opened a Business Critical Support SR with VMware on the question. I was told by VMware BCS that jumbo frames are NOT supported on ESXi 4 Update 1 VMkernel networks and a reference was made to the documentatation on page 54.

Our dedicated VMware onsite Engineer escalated and I was then told ESXi 4 Update 1 DOES support jumbo frames on VMkernel networks, making reference to Charu’s article.

Hey VMware, which is it? If this is a documentation mistake, why are you dragging your feet in getting the documentation updated two months after a VMware employee discovers the error and blogs it? Waiting for the next release of ESXi? Unacceptable! You update the public documentation as soon as you discover the error and be damned sure your BCS support Engineers know the right answer! Do you know how much companies pay for BCS? You owe your customers the correct answer. If misinformation comes as a result of a known documentation error, SHAME ON YOU! Architecture and design decisions are being made daily on this information or misinformation, which ever it may be.

Update 2/23/10: Toby Kraft (@vmwarewriter on Twitter) will be updating the documentation by next week. Thank you Toby!

Update 3/1/10: VMware has updated their documentation to reflect currently supported configurations. Thank you VMware (and Toby)!

5 comments »

Posted in Virtualization

Tags: Documentation ESXi Networking Rant VMware vSphere

Saturday Grab Bag

September 12th, 2009

Here’s a collection of quick hits I’ve been meaning to get to. Individually, their content is a bit on the short side for the length I normally like to write so I thought I’d throw them together in a single post and see how it comes out.

Tasks and Events List Lengths

First up is the listing of Tasks and Events in the vSphere Client. Have you ever started troubleshooting an issue in the vSphere client by looking at the Tasks or Events and the chronological listing of events doesn’t go back far enough to the date or time you’re looking for? Not finding the logs you’re looking for in the vSphere Client usually means you need to open a PuTTY session and start sifting through logs in /var/log/ or /var/log/vmware/ in the Service Console. The reason for this is that the vSphere Client, by default, is configured to tail the last 100 entries in the Tasks or Events list. You can find this setting in your vSphere Client by choosing “Edit|Client Settings” then choose the “Lists” tab:

Simply increase the value from 100 to whatever you’d like, with 1,000 being the highest allowable value. Notice that when this number is increased, you will immediately see more history. In other words, you don’t have to necessarily wait for time to pass and more historical events to accumulate to see the additional rows of information. Also note that this is a vSphere Client setting which is retained client side and applies to both vCenter Server and ESX(i) host connections.

Collecting diagnostic information for VMware products

Like any offering from a software or hardware vendor, VMware products aren’t perfect. During your VMware experience, you may run into a problem which requires the intervention of VMware support. More often than not, VMware is going to ask you to generate a support bundle which consists of a collection of diagnostic and configuration files and logs. Following this paragraph is a link to VMware KB1008524 which contains links to creating support bundles for various VMware products. Note that in some cases there are different methods for different versions of the same product. If you choose to create a VMware SR online, it is helpful to have created these log bundles in advance so you can attach them to the SR. If you’ve done VMware support long enough, you already know how to FTP log bundles to VMware after an SR number has been generated.

Collecting diagnostic information for VMware products

New VMware Update Manager won’t download ESX(i) patches

Scenario: You’ve built a new VMware vCenter Server in addition to a new VMware Update Manager Server (VUM). After properly configuring Update Manager as well as the necessary internet, proxy, baseline, and scheduled task settings, VUM proceeds to download Windows, Linux, and application patches, but it won’t download ESX(i) host patches. As I found out by trench experience, the cause is because no ESX(i) hosts have been added to the vCenter Server and thus no hosts are being managed by VUM. You need to add at least one ESX(i) host to vCenter Server before VUM will be triggered to suck down all the host updates. One might then ask why guest patches are being downloaded. The only answer I have for the inconsistent behavior is due the fact that ESX(i) host patches are downloaded from VMware, while guest OS and application patches are downloaded from a completely different source, Shavlik. The mechanics behind the download processes obviously differ between the two.

What vCenter Server is this ESX(i) host managed by?

Scenario: You administer a large VMware virtual infrastructure with many vCenter Servers. You need to manage or configure a host or cluster but haven’t the slightest idea what vCenter Server to connect to. You can easily find out by attempting a Virtual Infrastructure Client connection to the host in question. Shortly after providing the necessary host credentials, the IP address of the vCenter Server managing this host will be revealed:

Now in theory, you could establish a Virtual Infrastructure Client connection to the IP address, however, I don’t like this because it dirties up the cached connection list with IP addresses which are meaningless short of having them all memorized. I prefer to take it a step further by opening a Command Prompt and using the command ping -a <IP_address> to reveal the name of the vCenter Server managing the host:

The command above reveals jarjar.boche.mcse as the vCenter Server which is managing the ESX(i) host I was wanting to manage via the vCenter Server.

I’m sure a PowerShell expert will follow up with a script which makes this process easier but this a good example to follow if you don’t have PowerShell or the VI Toolkit (Power CLI) installed.

1 comment »

Posted in Virtualization

Tags: Documentation Update Manager vCenter Server VMware

Hidden Virtual CPU Limit Restriction in ESX 3.5

August 18th, 2009

Here’s something interesting to watch out for if you’re running ESX 3.5 Update 1 or newer clusters. In particular, clusters densely populated with running VMs or VMs with 2-way or 4-way vSMP.

Prior to ESX 3.5 Update 1, the supported and configured maximum number of vCPUs on a host was 128 by default. This meant that VMs totaling up to 128 vCPUs could be powered on within a single host.

With the release of ESX 3.5 Update 1, the supported maximum number of vCPUs on a host was raised to 192. This meant that VMs totaling up to 192 vCPUs could be powered on within a single host. Effectively, VMware is allowing higher consolidation ratios on a single host. However, according to KB article 1006393, ESX 3.5 Update 1 and newer hosts will still be configured to run a maximum of 128 vCPUs! Through my experience, this applies to both new installations of ESX 3.5 Update 1 and newer, as well as ESX 3.5 hosts that have been patched/remediated with Update 1 or newer.

So how does this impact a cluster? As I said in the beginning, you’ll run into problems on highly populated clusters or clusters with large numbers of VMs with vSMP CPUs enabled. You’ll see a few different but closely related scenarios:

VMs will not VMotion onto a host which would cause it to exceed a 128 running vCPU limit
DRS will not move running VMs onto a host which would cause it to exceed a 128 running vCPU limit
Maintenance Mode for a host will never complete if evacuation of the running VMs would cause all other hosts in the cluster to exceed a 128 running vCPU limit
HA will not power on VMs which would cause a host to exceed a 128 running vCPU limit
You will not be able to power on a VM which would cause a host to exceed a 128 running vCPU limit

To configure an ESX 3.5 Update 1 or newer host to support the maximum number of running vCPUs (192), follow the instructions in the KB article above which I will repeat here:

In the VI Client, go to the Configuration tab and choose Advanced Settings.

In the Advanced Settings window, change the value for Misc.RunningVCpuLimit to 192.

The increased maximum limit takes effect immediately and is retained after rebooting the host.

Repeat the steps above for each host.

VMware made a change for the better in vSphere. ESX 4.0 supports a maximum of 512 vCPUs and this is the way the host is configured in a default installation, thus, no hidden restriction as we find in ESX 3.5 Update 1 and newer.

Update 8/19/09: VMTN community member William Lam read this article and published a Perl script which will query a specific cluster and extract out the number of vCPU for the given cluster, each individual host and the advanced configuration Misc.RunningVCpuLimit set for each host. Thanks a lot William!!

9 comments »

Posted in Virtualization

Tags: Documentation ESX VMware

vSphere 4 Reference Card now available

August 10th, 2009

Forbes Guthrie has done it again! His wildly successful VI3 reference card is now available in vSphere format. Head over to his site, vReference, and download your copy today. Be sure to thank him for his hard work! I for one appreciate all that he does. Thanks Forbes and I look forward to meeting you in a few weeks.

1 comment »

Posted in Virtualization

Tags: Documentation VMware vSphere

VI3 ATDG Book Full Download Available 7/19/09

July 18th, 2009

I have it on good authority that the VMware Infrastructure 3 Advanced Technical Design Guide and Advanced Operations Guide book will be made fully available for download in .PDF format tomorrow. The authors over at vmguru.com had previously been releasing two chapters at a time (one chapter in each of the two sections of the book), but a decision has been made that the next release will include the entire book.

Watch for the release at vmguru.com and grab your copy. If by chance they don’t make the Sunday release date, give them a break, these authors are among the hardest working in the business. I’m sure they’ll have it up very soon. This is a very generous contribution to the virtualization community as the book is only about a year old. Kudos to Scott Herold, Ron Oglesby, and Mike Laverick.

2 comments »

Posted in Virtualization

Tags: Books Documentation ESX ESXi VMware

VMware ESX Configuration Maximums Comparison Matrix

July 7th, 2009

Some of the this blog’s readers may know that one of my favorite VMware documents is the Configuration Maximums. I’ve mentioned it a few times. Sid Smith over at Daily Hypervisor put together a very nice consolidation of configuration maximums for ESX3, ESX3.5, and vSphere (ESX4.0).

The .PDF is assembled in a matrix format making comparison of the various VMware hypervisors easy to differentiate, compare, and contrast. Sid, you read my mind. Well done. Thank you for the early birthday/Christmas present!

2 comments »

Posted in Virtualization

Tags: Documentation ESX VMware vSphere

VMware vSphere Cheat Sheet

April 22nd, 2009

I’m not exactly sure where this document came from – I was unable to locate it on the internet, but it looks to have been generated by VMware for the partner or sales channels. I’m an end user so I typically don’t have my hands on sales material. The document summarizes vSphere features, licensing, tiers, and more. It’s not marked VMware company confidential so I’m going to go ahead and post it. Hopefully I won’t find myself begging for forgiveness.

I love the virtualization product comparisons. There’s a lot of smoke in the air coming from all three major virtualization camps. I think the product comparison charts really help answer the questions “Why VMware?” “Why not MS or Citrix?” “Is VMware’s price point worth it?” You bet it is. The data below speaks for itself.

8 comments »

Posted in Virtualization

Tags: Documentation VMware vSphere

VMware documentation library updates

April 2nd, 2009

Quick note: In case you missed it (like I did), VMware has updated most of their VMware Infrastructure 3 documentation. If you’re a documentation junkie (like me), you’ll want to re-download all of VMware’s VI3 documentation. About 75% of the documents have new file names as well.

http://www.vmware.com/support/pubs/vi_pages/vi_pubs_35u2.html

2 comments »

Posted in Virtualization

Tags: Documentation ESX ESXi vCenter Server VirtualCenter VMware

Setup for Microsoft cluster service

April 1st, 2009

Setting up a Microsoft cluster on VMware used to be a fairly straight forward task with a very minimal set of considerations. Over time, the support documentation has evolved into something that looks like it was written by the U.S Internal Revenue Service. I was an Accountant in my previous life and I remember Alternative Minimum Tax code that was easier to follow than what we have today, a 50 page .PDF representing VMware’s requirements for MSCS support. Even with that, I’m not sure Microsoft supports MSCS on VMware. The Microsoft SVVP program supports explicit versions and configurations of Windows 2000/2003/2008 on ESX 3.5 update 2 and 3, and ESXi 3.5 update 3 but no mention is made regarding clustering. I could not find a definitive answer on the Microsoft SVVP program site other than the following disclaimer:

For more information about Microsoft’s policies for supporting software running in non-Microsoft hardware virtualization software please refer to http://support.microsoft.com/?kbid=897615. In addition, refer to http://support.microsoft.com/kb/957006/ to find more information about Microsoft’s support policies for its applications running in virtual environments.

At any rate, here are some highlights of MSCS setup on VMware Virtual Infrastructure, and by the way, all of this information is fair game for the VMware VCP exam.

Prerequisites for Cluster in a Box

To set up a cluster in a box, you must have:

* ESX Server host, one of the following:

* ESX Server 3 - An ESX Server host with a physical network adapter for the

service console. If the clustered virtual machines need to connect with external

hosts, then an additional network adapter is highly recommended.

* ESX Server 3i - An ESX Server host with a physical network adapter for the

VMkernel. If the clustered virtual machines need to connect with external

hosts, a separate network adapter is recommended.

* A local SCSI controller. If you plan to use a VMFS volume that exists on a SAN, you

need an FC HBA (QLogic or Emulex).

You can set up shared storage for a cluster in a box either by using a virtual disk or by

using a remote raw device mapping (RDM) LUN in virtual compatibility mode

(non‐pass‐through RDM).

When you set up the virtual machine, you need to configure:

* Two virtual network adapters.

* A hard disk that is shared between the two virtual machines (quorum disk).

* Optionally, additional hard disks for data that are shared between the two virtual

machines if your setup requires it. When you create hard disks, as described in this

document, the system creates the associated virtual SCSI controllers.

Prerequisites for Clustering Across Boxes

The prerequisites for clustering across boxes are similar to those for cluster in a box.

You must have:

* ESX Server host. VMware recommends three network adapters per host for public

network connections. The minimum configuration is:

* ESX Server 3 - An ESX Server host configured with at least two physical

network adapters dedicated to the cluster, one for the public and one for the

private network, and one network adapter dedicated to the service console.

* ESX Server 3i - An ESX Server host configured with at least two physical

network adapters dedicated to the cluster, one for the public and one for the

private network, and one network adapter dedicated to the VMkernel.

* Shared storage must be on an FC SAN.

* You must use an RDM in physical or virtual compatibility mode (pass‐through

RDM or non‐pass‐through RDM). You cannot use virtual disks for shared storage.

Prerequisites for Standby Host Clustering

The prerequisites for standby host clustering are similar to those for clustering across

boxes. You must have:

* ESX Server host. VMware recommends three network adapters per host for public

network connections. The minimum configuration is:

* ESX Server 3 - An ESX Server host configured with at least two physical

network adapters dedicated to the cluster, one for the public and one for the

private network, and one network adapter dedicated to the service console.

* ESX Server 3i - An ESX Server host configured with at least two physical

network adapters dedicated to the cluster, one for the public and one for the

private network, and one network adapter dedicated to the VMkernel.

* You must use RDMs in physical compatibility mode (pass‐through RDM).

You cannot use virtual disk or RDM in virtual compatibility mode

(non‐pass‐through RDM) for shared storage.

* You cannot have multiple paths from the ESX Server host to the storage.

* Running third‐party multipathing software is not supported. Because of this

limitation, VMware strongly recommends that there only be a single physical path

from the native Windows host to the storage array in a configuration of

standby‐host clustering with a native Windows host. The ESX Server host

automatically uses native ESX Server multipathing, which can result in multiple

paths to shared storage.

* Use the STORport Miniport driver for the FC HBA (QLogic or Emulex) in the

physical Windows machine.

	Cluster in a Box	Cluster Across Boxes	Standby Host Clustering
Virtual disks	Yes	No	No
Pass-through RDM (physical compatibility mode)	No	Yes	Yes
Non-pass-through RDM (virtual compatibility mode)	Yes	Yes	No

Caveats, Restrictions, and Recommendations

This section summarizes caveats, restrictions, and recommendation for using MSCS in

a VMware Infrastructure environment.

* VMware only supports third‐party cluster software that is specifically listed as

supported in the hardware compatibility guides. For latest updates to VMware

support for Microsoft operating system versions for MSCS, or for any other

hardware‐specific support information, see the Storage/SAN Compatibility Guide for

ESX Server 3.5 and ESX Server 3i.

* Each virtual machine has five PCI slots available by default. A cluster uses four of

these slots (two network adapters and two SCSI host bus adapters), leaving one

PCI slot for a third network adapter (or other device), if needed.

* VMware virtual machines currently emulate only SCSI‐2 reservations and do not

support applications using SCSI‐3 persistent reservations.

* Use LSILogic virtual SCSI adapter.

* Use Windows Server 2003 SP2 (32 bit or 64 bit) or Windows 2000 Server SP4.

VMware recommends Windows Server 2003.

* Use two‐node clustering.

* Clustering is not supported on iSCSI or NFS disks.

* NIC teaming is not supported with clustering.

* The boot disk of the ESX Server host should be on local storage.

* Mixed HBA environments (QLogic and Emulex) on the same host are not

supported.

* Mixed environments using both ESX Server 2.5 and ESX Server 3.x are not

supported.

* Clustered virtual machines cannot be part of VMware clusters (DRS or HA).

* You cannot use migration with VMotion on virtual machines that run cluster

software.

* Set the I/O time‐out to 60 seconds or more by modifying

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\

TimeOutValue.

The system might reset this I/O time‐out value if you recreate a cluster. You must

reset the value in that case.

* Use the eagerzeroedthick format when you create disks for clustered virtual

machines. By default, the VI Client or vmkfstools create disks in zeroedthick

format. You can convert a disk to eagerzeroedthick format by importing,

cloning, or inflating the disk. Disks deployed from a template are also in

eagerzeroedthick format.

* Add disks before networking, as explained in the VMware Knowledge Base article

at http://kb.vmware.com/kb/1513.

phew!

2 comments »

Posted in Virtualization

Tags: Cluster Documentation ESX ESXi Microsoft Virtualization VMware

VMware raises the bar on CPU consolidation ratio support

April 1st, 2009

VMware has updated its Configuration Maximums support document (one of my favorite most documents in the VMware document library). Most notable is the increase in number of supported virtual CPUs per core:

Previously, ESX and ESXi 3.5 Update 2 and earlier supported 8 virtual CPUs per core and in special cases, 11 virtual CPUs per core if the workloads were VDI
The new version of the document shows ESX and ESXi 3.5 Update 3 and later support 20 virtual CPUs per core across the board – with no special circumstances for VDI workloads

One thing to note however is the fact that the number of total virtual CPUs per host or total number of virtual machines per host did not change. They remain at 192 and 170 respectively.

So we’re not increasing the total number of VMs an ESX or ESXi host will support. VMware is saying they can support the same number of VMs and vCPUs on less physical CPU cores. This may be due to more powerful CPUs entering the market (such as the Intel Nehalem). Or maybe VMware is addressing customers who have traditionally light CPU workloads and need to reach higher CPU consolidation ratios. Or maybe it has something to do with blade servers or Cisco’s UCS (or Project California). At any rate, VMware is encouraging the virtualization of more with less. Maybe it’s an economy thing. Who knows. It’s good for us though since VMware still licenses by the socket and not the core. We can power 160 VMs with an 8 core box (dual quads or quad duals).

While we’re on the subject, is anyone coming close to 170 VMs per host? What’s the most impressive consolidation ratio you’ve seen? I’d like to hear about it. As in the Citrix world, I don’t think it’s a matter of “Do we have the hardware today to handle it?” – the answer is yes. It’s more the exposure of 170 VMs on a single host and do we want to go down that road.

4 comments »

Posted in Virtualization

Tags: Documentation ESX ESXi VMware

VI Toolkit Quick Reference Guide

March 14th, 2009

Virtu-Al (Alan Renouf) has posted a great two-page cheat sheet for the VMware VI Toolkit version 1.5.

This gem of a document is similar to VI3 card created by Forbes Guthrie over at vReference.com. Excellent job gentlemen!

While you’re at Virtu-Al’s site, check out all the sample code and scripts. Chances are you could implement one or more of these puppies in your environment to configure ESX or ESXi. Scripting is definitely one of the ways to become more efficient and agile and it’s a great way to ensure consistency across your environment. PowerShell and VI Toolkit is where’s it at. I think they are going to be here for a long time.

2 comments »

Posted in Virtualization

Tags: Documentation ESX ESXi Scripting VMware

Microsoft Performance Monitor tweaks

February 17th, 2009

Today I discovered the workarounds to a few issues in Microsoft Performance Monitor that have bugged me for quite a while (read: years).

Issue 1: Vertical lines are displayed in the Sysmon tool that obscure the graph view

Cause: This behavior occurs when there are more than 100 data points to be displayed in chart view.

Resolution: Microsoft KB article 283110

To enable or disable this behavior:

Start Regedit.exe.
Navigate to the following key:
HKEY_CURRENT_USER\Software\Microsoft\SystemMonitor
On the Edit menu, click New, and then click DWord Value.
Type the following value in the Name box:
DisplaySingleLogSampleValue
Set the value to 1 if you do not want to view the vertical line indicators, or set the value to 0, which is the default setting, to display the vertical indicators.

Result:

Issue 2: When looking at large numbers in Performance Monitor (Windows XP), comma separators do not exist thus making it difficult to interpret large numbers.

Cause: Microsoft

Resolution: Microsoft KB article 300884

Follow these steps, and then quit Registry Editor:

Click Start, click Run, type regedit, and then click OK.
Locate and then click the following key in the registry:
HKEY_CURRENT_USER\Software\Microsoft\SystemMonitor\
On the Edit menu, point to New, and then click DWORD Value.
Type DisplayThousandsSeparator, and then press ENTER.
On the Edit menu, click Modify.
Type 1, and then click OK.

Result:

Extra credit: Check out Microsoft KB article 281884 for one additional tweak that deals with viewing PIDs in Performance Monitor counters.

4 comments »

Posted in General

Tags: Documentation Microsoft Rant

Virtualization Wiki launched

February 11th, 2009

Rynardt Spies, proprietor of the VirtualVCP blog, has launched VI-Pedia, the Virtualization Open Wiki.

It looks like Rynardt has already begun populating the Wiki with links to VMware’s HCL information. I think the following information which I posted over at the Petri IT Knowledgebase would also prove to be useful on the Wiki:

Community-Supported Hardware/Software for VMware Infrastructure
http://www.vmware.com/resources/communitysupport/
In 2007, VMware began maintaining a web page of non-HCL hardware that works with VMware ESX. This is a list of hardware and software components that have been reported to work with VMware Infrastructure, either by the community or by the individual vendors themselves. Great for people trying to build a cheap lab out of dubious or whitebox hardware. If your hardware is not on the official VMware HCL, check this list to see if someone has reported that your particular piece of hardware works with ESX.

Additional Resources for Community-Supported Hardware/Software for VMware Infrastructure
http://www.vm-help.com/
http://www.vm-help.com/Whitebox_HCL.php
http://ultimatewhitebox.com/
http://www.vmweekly.com/articles/hardware_recommendations_to_build_cheap_esx_server/1/
http://www.mikedipetrillo.com/mikedvirtualization/2008/10/building-a-500-vmware-esxi-host.html