Posts Tagged ‘Security’

Father’s Day Fun With ESXi Annotations

June 19th, 2011

Tired of the same old DCUI look of your ESXi host?

SnagIt Capture

Change it with your own custom Annotation (found under Host|Configuration|Software|Advanced Settings):

SnagIt Capture

Viola!

SnagIt Capture

Clearly this is more fun than any one person should be allowed to have.

Clear the Annotation field to restore the original ESXi look.

Make an ESX Firewall Rule Manageable in the vSphere Client

June 25th, 2010

Make an ESX Firewall Rule Manageable in the vSphere Client.  To do so, you essentially need to create a new service in the firewall configuration XML file.

Open the file /etc/vmware/firewall/services.xml
Scroll to the bottom & note the last Service ID #
Copy an existing service section as a template (ie. faultTolerance)
Paste as new following proper XML formatting
Increment the Service ID # by 1 ensuring it’s unique
Customize to fit your new inbound/outbound port rule
Save and exit
Services do not need to be restarted

As an example, I took :

<service id=’0031′>
    <id>faultTolerance</id>
    <rule id=’0000′>
      <direction>outbound</direction>
      <protocol>tcp</protocol>
      <port type=’dst’>80</port>
    </rule>
  </service>

and created a new service like so:

<service id=’0033′>
    <id>CoolFirewallRule</id>
    <rule id=’0000′>
      <direction>outbound</direction>
      <protocol>tcp</protocol>
      <port type=’dst’>12345</port>
    </rule>
  </service>

The result is a firewall rule named CoolFirewallRule which can be toggled via the vSphere Client:

 6-22-2010 11-13-39 PM

Two Quick Announcements

April 19th, 2010

I’ve got several blog posts in the queue but unfortunately I haven’t had time to get them cranked out yet.  In the interim, here are a couple of hot items I wanted to help spread the word on.

There’s a 45 minute VMware certification podcast coming up early this Wednesday morning.  APAC Vitalization Roundtable (Certifications & the VCDX path).  Podcast guests are Andrew Mitchell, Duncan Epping, and Alastair Cooke.  The Talkshoe podcast is scheduled for the following time:
EDT (USA) – 7AM
PDT (USA) – 4AM
Perth (Australia) – 7PM
Hong Kong (Hong Kong) – 7PM
Kuala Lumpur (Malaysia) – 7PM
Tokio (Japan) – 8PM
Auckland (New Zealand) – 11PM
London (UK) – 12Noon
 

The vSphere 4.0 Hardening Guide, a collaborative effort which has been under development for several months with a few interim beta distributions, has been finalized and released.  The document is presented in a flexible format in that a few different levels of hardening can be used as low to high risk boundaries.  The user then ultimately decides on the appropriate hardening level depending on the level of risk the user is comfortable with assuming.The guide covers several key VMware Virtual Infrastructure areas.  To quote VMware’s announcement: 

Overall, there are more than 100 guidelines, with the following major sections: 

  • Introduction
  • Virtual Machines
  • Host (both ESXi and ESX)
  • vNetwork
  • vCenter
  • Console OS (for ESX only)

If you’ve recently deployed vSphere or if you are about to, check out these hardening guides to help secure your datacenter.  You can’t beat the price – free!

New ESX(i) 3.5 security patch released; scenarios and installation notes

April 11th, 2009

On Friday April 10th, VMware released two patches:

Both address the same issue:

A critical vulnerability in the virtual machine display function might allow a guest operating system to run code on the host. The Common Vulnerabilities and Exposures Project (cve.mitre.org) has assigned the name CVE-2009-1244 to this issue.

Hackers must love vulnerabilities like this because they can get a lot of mileage out of essentially a single attack. The ability to execute code on an ESX host can impact all running VMs on that host.

Although proper virtualization promises isolation, the reality is that no hardware or software vendor is perfect and from time to time we’re going to see issues like this. Products are under constant attack from hackers (both good and bad) to find exploits. In virtualized environments, it’s important to remember that guest VMs and guest operating systems are no different than their physical counterparts in that they need to be properly protected from the network. That means adequate virus protection, spyware protection, firewalls, encryption, packet filtering, etc.

This vulnerability in VMware ESX and ESXi is really a two factor attack. In order to compromise the ESX or ESXi host, the guest VM must first be vulnerable to compromise on the network to provide the entry point to the host. Once the guest VM is compromised, the next step is to get from the guest VM to the ESX(i) host. Hosts without the patch will be vulnerable to the next attack which we know from reading above will allow who knows what code to be executed on the host. If the host is patched, we maintain our guest isolation and the attack stops at the VM level. Unfortunately, the OS running in the guest VM is still compromised, again highlighting the need for adequate protection of the operating system and applications running in each VM.

The bottom line is this is an important update for your infrastructure. If your ESX or ESXi hosts are vulnerable, you’ll want to get this one tested and implemented as soon as possible.

I installed the updates today in the lab and discovered something interesting that is actually outlined in both of the KB articles above:

  • The ESXi version of the update requires a reboot. Using Update Manager, the patch process goes like this: Remediate -> Maintenance Mode -> VMotion VMs off -> Patch -> Reboot -> Exit Maintenance Mode. The duration of installation of the patch until exiting maintenance mode (including the reboot in between) took 12 minutes.
  • The ESX version of the update does not require a reboot. Using Update Manager, the patch process goes like this: Remediate -> Maintenance Mode -> VMotion VMs off -> Patch -> Exit Maintenance Mode. The duration of installation of the patch until exiting maintenance mode (with no reboot in between) took 1.5 minutes.

Given reboot times of the host, patching ESX hosts goes much quicker than patching ESXi hosts. Reboot times on HP Proliant servers aren’t too bad but I’ve been working with some powerful IBM servers lately and the reboot times on those are significantly longer than HP. Hopefully we’re not rebooting ESX hosts on a regular basis so with that in mind, reboot times aren’t a huge concern, but if you’ve got a large environment with a lot of hosts requiring reboots, the reboot times are going to be cumulative in most cases. Consider my environment above. A 6 node ESXi cluster is going to take 72 minutes to patch, not including VMotions. A 6 node ESX cluster is going to take 9 minutes to patch, not including VMotions. This may be something to really think about when weighing the decision of ESX versus ESXi for your environment.

Update: One more item critical to note is that although the ESX version of the patch requires no reboot, the patch does require three other patches to be installed, at least one of which requires a reboot.  If you already meet the requirements, no reboot will be required for ESX to install the new patch.

In closing, while we are on the subject of performing a lot of VMotions, take a look at a guest blog post from Simon Long called VMotion Performance. Simon shows us how to modify VirtualCenter (vCenter Server) to allow more simultaneous VMotions which will significantly cut down the amount of time spent patching ESX hosts in a cluster.

Three VirtualCenter security tips Windows administrators should know

January 15th, 2009

Good morning!  I’d like to take the opportunity to talk a bit about something that has been somewhat of a rock in my shoe as a seasoned Windows administrator from the NT 3.5 era:  The VirtualCenter (vCenter Server, VirtualCenter Management Server, VCMS, VC, etc.) security model, or more accurately, its unfamiliar mechanics that can catch Windows administrators off guard and leave them scratching their heads.

Tip #1: The VCMS security model revolves around privileges, roles, and objects.  The more than 100 privileges define rights, roles are a collection of privileges, and roles are assigned to objects which are entities in the virtual infrastructure as shown in the diagram borrowed below:

1-15-2009 11-24-45 AM

Windows administrators will be used to the concept of assigning NTFS permissions to files, folders, and other objects in Active Directory.  It is very common for Windows objects to contain more than one Access Control Entry (ACE) which can be a group (such as “Accounting”, “Marketing”, etc.) or an explicit user (such as “Bob”, Sally”, etc.)  The same holds true for assigning roles to object in VC.

In some instances, which are not uncommon at all, a user may be granted permission to an object by way of more than one ACE.  For example, if both the Accounting and Marketing groups were assigned rights, and Sally was a member of both those groups, Sally would have rights to the object through both of those groups.  Using this same example, if the two ACEs defined different permissions to an object, the end result is a cumulative, so long as the ACE doesn’t contain “deny” which is special:  Sally would have the combined set of permissions.  The same holds true in VC.

Let’s take the above example a step further.  In addition to the two groups, which Sally is a member of, being ACLd to an object, now let’s say Sally’s user account object itself is an explicit ACE in the ACL list.  In the Windows world, the effect is Sally’s rights are still cumulative combining the three ACEs.  This is where the fork in the road lies in the VirtualCenter security model.  Roles explicitly assigned to a user object trump all other assigned or inherited permissions to the same object.  If the explicit ACE defines less permissions, the effective result is Sally will have less permissions than what her group membership would have provided.  If the explicit ACE defines more permissions, the effective result is Sally will have more permissions than what her group membership would have provided.  This is where Windows based VC administrators will be dumbfounded when a user suddenly calls with tales of things gray’d out in VirtualCenter, not enough permissions, etc.  Of course the flip side of the coin is a junior administrator suddenly finds themselves with cool new options in VC.  “Let’s see what this datastore button does”

Moral of the story from a real world perspective:  Assigning explicit permissions to user accounts in VC without careful planning will yield somewhat unpredictable results when inheritance is enabled (which is typical).  To take this to extremes, assigning explicit permissions to user accounts in VC, especially where inheritance in the VC hierarchy is involved, is a security and uptime risk when a user ends up with the wrong permissions accidentally.  For security and consistency purposes, I would avoid assigning permissions explicitly to user accounts unless you have a very clear understanding of the impacts currently and down the road.

Tip #2: Beware the use of the built in role Virtual Machine Administrator.  It’s name is misleading and the permissions it has are downright scary and not much different than the built in Administrator role.  For instance, the Virtual Machine Administrator role:  can modify VC and ESX host licensing, has complete control over the VC folder structure, has complete control over Datacenter objects, has complete control over datastores (short of file management), can remove networks, has complete control over inventory items such as hosts and clusters.  This list goes on and on.  I have three words:  What The Hell?!  I don’t know – the way my brain works is those permissions stretch well beyond the boundaries of what I would delegate for a Virtual Machine Administrator.

Moral of the story from a real world perspective:  Use the Virtual Machine Administrator role with extreme caution.  There is little disparity between the Administrator role and the Virtual Machine Administrator role, minus some items for Update Manager and changing VC permissions themselves. Therefore, any user who has the Virtual Machine Administrator role is practically an administrator.  The Virtual Machine Administrator role should not be used unless you have delegations that would fit this role precisely.  Another option would be clone the role and strip some of the more datacenter impactful permissions out of it.

Tip #3: Audit your effective VirtualCenter permissions on a regular basis, especially if you have large implementation with many administrators “having their hands in the cookie jar” so to speak.  If you use groups to assign roles in VC, then that means you should be auditing these groups as well (above and beyond virtualization conversations, administrative level groups should be audited anyway as a best practice).  This whitepaper has a nice Perl script for dumping VirtualCenter roles and permissions using the VMware Infrastructure Perl Toolkit.  Use of the script will automate the auditing process quite a bit and help transform a lengthy mundane task into a quicker one.  While you’re at it, it wouldn’t be a bad idea to periodically check tasks and events to see who is doing what.  There should be no surprises there.

Moral of the story from a real world perspective:  Audit your VirtualCenter roles and permissions.  When an unexpected datacenter disaster occurs from users having elevated privileges, one of the first questions to be asked in the post mortem meeting will be what your audit process is.  Have a good answer prepared.  Even better, avoid the disaster and down time through the due diligence of auditing your virtual infrastructure security.

For more information about VirtualCenter security, check out this great white paper or download the .pdf version from this link.  Some of the information I posted above I gathered from this document.  The white paper was written by Charu Chaubal, a technical marketing manager at VMware and Ph.D. in numerical modeling of complex fluids, with contributions from Doug Clark, and Karl Rummelhart.

If VirtualCenter security talk really gets your juices flowing, you should check out a new podcast launched by well known and respected VMTN community member/moderator and book author Edward Haletky that starts today called Virtualization Security Round Table.  It is sure to be good!

VMware VI network communications and port usage diagram

November 6th, 2008

Nigel Metheringham has taken the information I posted here and updated it with information from page 179 of the VMware ESX Server 3 Configuration Guide

The result is a current diagram of VMware Virtual Infrastructure network communications and port usage which applies to both ESX and ESXi.  Nigel sent me the updated document via email so that I can update the information on the VMware Communities, however, they are currently unavailable due to planned maintenance so in the mean time I’m making the document available here.  Thank you Nigel!

Download from the link below:

vmware_network_ports