Cloud butt kicking

March 29th, 2009 by jason No comments »

Mike DiPetrillo wrote a nice piece yesterday entitled The Cloud is Kicking My Butt. This has helped put my mind at ease. Those whom I talk to might recall that I’ve been somewhat behind the learning curve on “the cloud”. To say my mind was blown at VMworld 2008 would have been an understatement with all of the cloud buzz and announcements.

It’s comforting to know that I’m in good company. In Mike’s article, he describes his new endeavor in 2009 as VMware’s Global Cloud Architect. Mike has talked to literally hundreds of people with mixed backgrounds and agendas about the cloud and he has come up with a few findings thus far:

  1. Cloud means 1,000 different things to 1,000 different people
  2. Everyone wants cloud today
  3. No one trusts external clouds and yet everyone wants to use them
  4. Absolutely everyone is ignorant on cloud
  5. There are only about 20 people in the world right now that truly “get it”

I can relate to bullets 1 and 4 above. And of the 20 people in the world who are cloud gurus… I’m not one of ’em. I’ve made some progress since fall of last year though. Simplified, I understand cloud is really no more than a collection of service offerings with special characteristics like portability, scalability, flexibility, high availability, compatibility, security, etc. It’s this very list of what the cloud is that people like me struggle with – because not everyone is going by the same list. The cloud means different things to different people. It’s going to be interesting to watch the when, where, what, and hows of inter cloud compatibility unfold. Without established standards I doubt we’d ever get there. We’re going to need a lot of cooperation between all vendors also, not just the key ones.

I visited with a VMware SE last week and heard one of the best, yet simple definitions of cloud yet: A Cluster Boundary. If you think about that from a VMware infrastructure perspective, it’s easy. VMs are servers, desktops, and applications that move about in the cluster. They are comprised of shared storage, networks, memory, and cpu. They are going to have isolation in most cases where needed and will be secure. Hardware independence makes them compatible with other hosts in the infrastructure that may not be the same exact make and model. It also affords them the ability to float to these other hosts in the cluster using technologies like VMotion, DRS, sVMotion, HA, and FT. But the cluster defines the boundaries of their mobility and thus it defines “a cloud”. The technologies and collaborative initiatives of tomorrow will be what make these clouds compatible and extensible so that are boundaries of the cloud are much much larger. In fact, in a perfect cloud world where all cloud components are compatible and seamless, there are no boundaries. It will be like space, where you have the ability to float wherever you’d like. For a fee or subscription of course.

The cloud is a magnificent undertaking. There are efforts involved which I don’t think we’ve even come close to seeing since the ratification of the TCP/IP protocol. I applaud people like Mike that have the stamina and drive to tackle this great initiative.

Mike concludes with:

Cloud is in flux. Standards don’t really exist. Everyone is labeling everything from toaster ovens to BMWs as “cloud enabled”. The information I share can and will change. Some of it will become obsolete. Some of it may seem very scary like no one has their act together. That’s just the nature of cloud at the moment. Everyone in the industry is stumbling through this together. In the end it’s all going to be worth it. For now though sit back, keep reading the posts, and prepare to get your butt kicked by the cloud.

I don’t think he could be more right if he tried. BTW, congrats on your new gig Mike! Go get ’em!

Anti-affinity rules are not honored in cluster with more than 2 virtual machines

March 27th, 2009 by jason No comments »

We can put a man on the moon and we can hot migrate virtual machines with SMP and gigs of RAM, but we can’t create anti-affinity rules with three or more VMs. This has been a thorn in my side since 2006, long before I requested it fixed in February 2007 on the VMTN Product and Feature Suggestions forum.

VMware updated KB article 1006473 on 3/26 outlining anti-affinity rule behavior when using three or more VMs:

“This is expected behavior, as anti affinity rules can be set only for 2 virtual machines.

When a third virtual machine is added any rule becomes disabled (with 2.0.2 or earlier).

There has been a slight change in behavior with VirtualCenter 2.5, wherein input validation occurs, where a third virtual machine added produces a warning message indicating a maximum of two virtual machines only can be added to this rule.

To workaround this, create more rules to cover all of the combinations of virtual machines.

For example, create rules for (VM1 & VM2), then (VM2 & VM3), and (VM1 & VM3).”

That last sentence is what has been burning my cookies for the longest time. In my last environment, I had several NLB VMs which could not be on the same host for load balancing and redundancy purposes. Rather than create a minimum amount of rules to intelligently handle all of the VMs, I was left with no choice but to create several rules for each potentially deadly combination.

Work harder, not smarter. Come on VMware.

VLC media player – free

March 25th, 2009 by jason No comments »

I came across some video training to preview for which I did not have an application that will play them back. Stu over at Vinternals suggested VLC media player which is a free piece of software that plays most audio and video formats.

Stu was right as rain. I did not recognize the file extension of the video files, and the file extension was not a registered type that VLC would natively support, however, VLC had no problem detecting the video format and then opening and playing back the file.

VLC provides the following charts showing their support for the various formats and platforms:

3-25-2009 11-44-48 PM

3-25-2009 11-45-19 PM

VMware releases Cost Per Application Calculator

March 23rd, 2009 by jason No comments »

On Monday morning 8am EDT, VMware will have released their Cost Per Application Calculator.  The tool was designed to accurately estimate and compare acquisition cost of a virtualization solution.  VMware has labeled this metric Cost Per Application.  Many market analysts, competitors, and potential customers have been focusing on the sheer cost of purchasing VMware licensing and support without performing the time consuming analysis that would reflect VMware’s complete cost effectiveness.  This calculator was designed to address that.  Goals outlined for the calculator included:

  • Simlified yet accurate metric to evaluate upfront costs only
  • Focus on easily quantifiable cost items that can be entered into the calculator, not indirect savings coming from things like improved productivity or operational efficiency
  • Present a cost comparison between VMware and competitor products using cost per application

VMware’s calculation for Cost Per Application is:

  • Virtualization software
    • Licenses for virtualization software

+

  • Hardware
    • Servers
    • Networking
    • Storage
    • Electricity
    • Datacenter space

+

  • VM Guest OS
    • Licenses for virtual machine operating systems

+

  • Management
    • Management software
    • Management servers and other infrastructure cost
    • Database software and servers

÷

  • Number of Apps Virtualized

=

  • $$ Cost Per Application $$

Note, the Cost Per Application Calculator is not meant to be a replacement of VMware’s TCO and ROI Calculator which is always the optimal methodology to evaluate a software invstment with a complete cost/savings analysis.  VMware’s TCO and ROI Calculator accounts for more items such as multi year views, depreciation, and amortization which the Cost Per Application Calculator does not .

Active Directory authentication with VMware ESX

March 21st, 2009 by jason No comments »

Integrating Microsoft Active Directory authentication in the ESX Service Console used to be somehwhat of a daunting task.  Today, however, the steps have been greatly condensed and I find it a cinch.  It basically boils down to two steps for each ESX host you wish to integrate AD with (which can be scripted I might add):

  1. Create a local user account on the ESX host for each AD user account you would like to authenticate
    • /usr/sbin/useradd yourusername
  2. Execute an authentication configuration command on the ESX host to be integrated with AD
    • /usr/sbin/esxcfg-auth –enablead –addomain yourdomain.com –addc yourdc.yourdomain.com –krb5realm=yourdomain.com –krb5kdc yourdc.yourdomain.com –krb5adminserver yourdc.yourdomain.com –enablekrb5
      • Assuming your AD environment has multiple domain controllers, consider adding redundant –addc yourdc.yourdomain.com parameters to the command line above in case one of your DCs become unavailable on the network

Be sure to repeat the steps above for each ESX host you wish to integrate AD with.  As new VI administrators come and go in your environment you’ll need to repeat step 1 above to add new administrator accounts on each ESX host.  For administrators that have terminated, you’ll need to remove their account (and home directory if it exists) from each ESX host using the console command /usr/sbin/userdel -r username.  -r specifies the removal of the respective home directory if it exists.

For a thorough explanation of the esxcfg-auth command, execute the command man esxcfg-auth in the Service Console.

Warning:  One thing to watch out for would the existance of a root account in AD in which you are not the owner of.  By implementing AD authentication, a root account in AD is going to be granted root level Service Console access on the ESX host!  Take the necessary precautions here.

Storage block size and alignment

March 20th, 2009 by jason No comments »

Steve Chambers posted version 2 of the Storage block size and alignment document over at the VIOPS (VMware Virtual Infrastructure Operations) site. At seven pages, it is both a short and a GREAT read.

For those not familiar with VMFS and VM guest alignment, I’ll summarize:

VMFS Alignment

  1. Unaligned volumes result in track crossing and additional I/O penalties in the form of latency and throughput which may or may not be noticeable in your environment (it depends)
  2. To verify whether or not your VMFS volumes are aligned, run the fdisk -lu command at the console
  3. VMFS volumes created with the Virtual Infrastructure Client (vSphere Client) are automatically aligned since it automatically align the volume along the 64KB boundary so no need to worry about the sub bullets in #2 above.
  4. NFS datastores are not concerned with VMFS alignment as they are not block VMFS datastores
  5. Alternatively, VMFS volumes can be aligned by following a series of fdisk commands manually which will destroy data on the volume (definitely not preferred)
  6. VMFS block size only determines maximum file size on the VMFS volume. VMFS block size does not play even a remotely significant performance role.  There are a number of expert blog articles which debate this.

VM Guest Alignment

  1. To verify whether or not your VM guest virtual disks are aligned, check the partition offset value
    • Aligned virtual disks will have a partition offset value evenly divisible by 4,096 (ie. 65,536 or 1,048,576 which is a default for Windows Server 2008)
    • Non-aligned virtual disks will have a partition offset value not evenly divisible by 4,096 (ie. 32,256 which is a default for Windows XP and Server 2003)
  2. Due to the destructive nature of the alignment procedures, alignment is always performed before data is placed on the volume
  3. Alignment in Linux guests is performed using an almost identical series of fdisk commands listed in a previous bullet
  4. Alignment in Windows guests is performed using diskpart.exe
  5. Although guest alignment is data destructive, guest alignment can be performed after the guest OS is installed because the document recommends that alignment of the OS partition is unnecessary; only align the data partitions before data is placed on them.  **see update below**

Alignment is most often going to be labor intensive and thus will have diminishing returns. This will especially be true if your environment has already been built and you need to align after the fact. Environments in the planning stages and not yet built will be among the best candidates for alignment right out of the gate. Whatever stage you are at, updating guest VM templates with alignment wouldn’t be a bad idea. Alignment of one image will pay dividends, whether noticeable or not, over and over as that template is deployed throughout the infrastructure.

Update: NetApp released a few scripts that will not only automate the verification and alignment processes at the guest VM OS level, the script will align the guest OS without destroying data. The one exception I ran into was with a Citrix VM that had remapped drives. CTXGINA.DLL got real cranky. The scripts are:

  • mbrscan – Scans the -flat.vmdk file for alignment
  • mbralign – Makes a backup of the .vmdk and creates a newly aligned .vmdk

See also:  NetApp – Storage Nuts & Bolts: mbrscan/mbralign

3-20-2009 1-24-50 PM

Other recommended reading:

Recommendations for Aligning VMFS Partitions

Performance Best Practices for VMware vSphere 4.1

MobilePress caused 55,000+ files in c:\windows\temp

March 19th, 2009 by jason No comments »

A while after installing the MobilePress 1.0.3 plugin for WordPress, my IIS server locked up.  I rebooted it and all was well.  A while later, it locked up again.  Upon further investigation, I found 55,000+ files in the c:\windows\temp\ folder and new files were popping in there at a rate of a few per minute.

Each of the 55,000 files looked like:

sess_1dq5436rb4m9b399cojhnmitd1

sess_3meinb58v9oqra5ia0869pqig6

sess_5hbicsnrt0hn1qj9lc5q9n7g30

where the prefix of sess_ is common but the rest is random.

Using Sysinternals procmon.exe, I was able to identify right away that the process responsible for creating the files was w3wp.exe which pointed me to IIS.  However, I wasn’t sure why IIS would begin doing this after being stable for a long time.

Searches on the internet said the files were being generated by PHP and indicated new user sessions as visitors hit my blog.  That helped confirm the fact that these were coming from IIS and the blog but still no tell tale reason as to why all of the sudden.

Then I opened up one of the files and it showed:
SESS_MOBILE_BROWSER|s:6:”mobile”;SESS_MOBILE_ACTIVE|b:0;SESS_MOBILE_THEME|s:7:”default”;

That was enough to jog my memory that I had recently installed the MobilePress plugin.

Removing the plugin immediately resolved the issues and the temp files are no longer created.