Good morning! I’d like to take the opportunity to talk a bit about something that has been somewhat of a rock in my shoe as a seasoned Windows administrator from the NT 3.5 era: The VirtualCenter (vCenter Server, VirtualCenter Management Server, VCMS, VC, etc.) security model, or more accurately, its unfamiliar mechanics that can catch Windows administrators off guard and leave them scratching their heads.
Tip #1: The VCMS security model revolves around privileges, roles, and objects. The more than 100 privileges define rights, roles are a collection of privileges, and roles are assigned to objects which are entities in the virtual infrastructure as shown in the diagram borrowed below:
Windows administrators will be used to the concept of assigning NTFS permissions to files, folders, and other objects in Active Directory. It is very common for Windows objects to contain more than one Access Control Entry (ACE) which can be a group (such as “Accounting”, “Marketing”, etc.) or an explicit user (such as “Bob”, Sally”, etc.) The same holds true for assigning roles to object in VC.
In some instances, which are not uncommon at all, a user may be granted permission to an object by way of more than one ACE. For example, if both the Accounting and Marketing groups were assigned rights, and Sally was a member of both those groups, Sally would have rights to the object through both of those groups. Using this same example, if the two ACEs defined different permissions to an object, the end result is a cumulative, so long as the ACE doesn’t contain “deny” which is special: Sally would have the combined set of permissions. The same holds true in VC.
Let’s take the above example a step further. In addition to the two groups, which Sally is a member of, being ACLd to an object, now let’s say Sally’s user account object itself is an explicit ACE in the ACL list. In the Windows world, the effect is Sally’s rights are still cumulative combining the three ACEs. This is where the fork in the road lies in the VirtualCenter security model. Roles explicitly assigned to a user object trump all other assigned or inherited permissions to the same object. If the explicit ACE defines less permissions, the effective result is Sally will have less permissions than what her group membership would have provided. If the explicit ACE defines more permissions, the effective result is Sally will have more permissions than what her group membership would have provided. This is where Windows based VC administrators will be dumbfounded when a user suddenly calls with tales of things gray’d out in VirtualCenter, not enough permissions, etc. Of course the flip side of the coin is a junior administrator suddenly finds themselves with cool new options in VC. “Let’s see what this datastore button does”
Moral of the story from a real world perspective: Assigning explicit permissions to user accounts in VC without careful planning will yield somewhat unpredictable results when inheritance is enabled (which is typical). To take this to extremes, assigning explicit permissions to user accounts in VC, especially where inheritance in the VC hierarchy is involved, is a security and uptime risk when a user ends up with the wrong permissions accidentally. For security and consistency purposes, I would avoid assigning permissions explicitly to user accounts unless you have a very clear understanding of the impacts currently and down the road.
Tip #2: Beware the use of the built in role Virtual Machine Administrator. It’s name is misleading and the permissions it has are downright scary and not much different than the built in Administrator role. For instance, the Virtual Machine Administrator role: can modify VC and ESX host licensing, has complete control over the VC folder structure, has complete control over Datacenter objects, has complete control over datastores (short of file management), can remove networks, has complete control over inventory items such as hosts and clusters. This list goes on and on. I have three words: What The Hell?! I don’t know – the way my brain works is those permissions stretch well beyond the boundaries of what I would delegate for a Virtual Machine Administrator.
Moral of the story from a real world perspective: Use the Virtual Machine Administrator role with extreme caution. There is little disparity between the Administrator role and the Virtual Machine Administrator role, minus some items for Update Manager and changing VC permissions themselves. Therefore, any user who has the Virtual Machine Administrator role is practically an administrator. The Virtual Machine Administrator role should not be used unless you have delegations that would fit this role precisely. Another option would be clone the role and strip some of the more datacenter impactful permissions out of it.
Tip #3: Audit your effective VirtualCenter permissions on a regular basis, especially if you have large implementation with many administrators “having their hands in the cookie jar” so to speak. If you use groups to assign roles in VC, then that means you should be auditing these groups as well (above and beyond virtualization conversations, administrative level groups should be audited anyway as a best practice). This whitepaper has a nice Perl script for dumping VirtualCenter roles and permissions using the VMware Infrastructure Perl Toolkit. Use of the script will automate the auditing process quite a bit and help transform a lengthy mundane task into a quicker one. While you’re at it, it wouldn’t be a bad idea to periodically check tasks and events to see who is doing what. There should be no surprises there.
Moral of the story from a real world perspective: Audit your VirtualCenter roles and permissions. When an unexpected datacenter disaster occurs from users having elevated privileges, one of the first questions to be asked in the post mortem meeting will be what your audit process is. Have a good answer prepared. Even better, avoid the disaster and down time through the due diligence of auditing your virtual infrastructure security.
For more information about VirtualCenter security, check out this great white paper or download the .pdf version from this link. Some of the information I posted above I gathered from this document. The white paper was written by Charu Chaubal, a technical marketing manager at VMware and Ph.D. in numerical modeling of complex fluids, with contributions from Doug Clark, and Karl Rummelhart.
If VirtualCenter security talk really gets your juices flowing, you should check out a new podcast launched by well known and respected VMTN community member/moderator and book author Edward Haletky that starts today called Virtualization Security Round Table. It is sure to be good!
I recently finished reading the book VMware VI3 Implementation and Administration by Eric Siebert (ISBN-13: 978-0-13-700703-5). VMware VI3 Implementation and Administration was a very enjoyable read. I don’t mean to sound cliché but for me it was one of those books that is hard to put down. Released in May of 2009, along with the next generation of VMware IV (vSphere), the timing of its arrival to market probably could have been better, but better late than never. Datacenters will be running on VI3 for quite some time. With that in mind, this book provides a tremendous amount of value and insight. I can tell that Eric put a lot of time and research into this book; the quality of the content shows. Much of the book was review for me, but I was still able to pick up bits and pieces here and there I wasn’t aware of, as well as some fresh perspective and new approaches to design, administration, and support.
