Posts Tagged ‘Windows’

Q: What’s your Windows template approach?

November 7th, 2010

Once upon a time, I was a Windows Server administrator.  Most of my focus was on Windows Server deployment and management. VMware virtualization was a large interest but my Windows responsibilities dwarfed the amount of time I spent with VMware.  One place where these roads intersect is Windows templates.  Because a large part of my job was managing the Windows environment, I spent time maintaining “the perfect Windows template”.  Following were the ingredients I incorporated:

Adobe Acrobat Reader Advanced Find & Replace Beyond Compare
Diskeeper MS Network Monitor MS Resource Kits
NTSEC Tools Latest MS RDP Client Symantec Anti-Virus CE
MS UPHClean VMware Tools Windows Admin Pack
Windows Support Tools Winzip Pro Sysinternals Suite
Windows Command Console BGINFO CMDHERE
Windows Perf Advisor MPS Reports GPMC


Remote Desktop enabled Remote Assistance disabled Pagefile
Complete memory dump DIRCMD=/O env. variable PATH tweaks
taskmgr.exe in startup, run minimized SNMP Desktop prefs.
Network icon in System Tray Taskbar prefs.  
C: 12GB D: 6GB  
Display Hardware acceleration to Full*    
* = if necessary    


VMware virtualization is now and has been my main focus going on two years.  By title, I’m no longer a Windows Server administrator and I don’t care to spend a lot of time worrying about what’s in my templates.  I don’t have to worry about keeping several applications up to date.  In what I do now, it’s actually more important to consistently work with the most generic Windows template as possible.  This is to ensure that projects I’m working with on the virtualization side of things aren’t garfed up by any of the 30+ changes made above.  Issues would inevitably appear and each time I’d need to counter productively deal with the lists above as possible culprits.  As such, I now take a minimalist approach to Windows templates as follows:

VMware Tools


C: 20GB VMXNET3 vNIC Activate Windows
wddm_video driver* Disk Alignment Display Hardware acceleration to Full*
* = if necessary    


In large virtualized environments, templates may be found in various repositories due to network segmentation, firewalls, storage placement, etc.  As beneficial as templates are, keeping them up to date can become a significant chore and the time spent doing so eats away at the time savings benefit which they provide.  Deployment consistency is key in reducing support and incident costs but making sure templates in distributed locations are consistent is not only a chore, but it is of paramount importance.  If this is the scenario you’re fighting, automated template and/or storage replication is needed.  Another solution might be to get away from templates altogether and adopt a scripted installation which is another tried and true approach which provides automation and consistency, but without the hassle of maintaining templates.  The hassle in this case isn’t eliminated completely.  It’s shifted into other areas such as maintaining PXE boot services, maintaining PXE images, and maintaining post build/application installation scripts.  I’ve seen large organizations go the scripted route in lieu of templates.  One reason could simply be that scripted virtual builds are strategically consistent with the organization’s scripted physical builds.  Another could be the burden of maintaining templates as I discussed earlier.  Is this a hint that templates don’t scale in large distributed environments?

Do you use templates and if so, what is your approach in comparison to what I’ve written about?

vCenter Server Linked Mode Configuration Error

October 23rd, 2010

As of vCenter Server 4.1, VMware supports Windows Server 2008 R2 as a vCenter platform (remember 2008 R2 is 64-bit only).  With this, I expect many environments will be configured with vCenter Server on Microsoft’s newest Server operating system.

While working in the lab with vCenter Server 4.1 on Windows Server 2008 R2, I ran into an issue configuring Linked Mode via the vCenter Server Linked Mode Configuration shortcut.  Error 28035. Setup failed to copy LDIFDE.EXE from System folder to ‘%windir%\ADAM’ folder.

SnagIt Capture

After no success in relaxing Windows NTFS permissions, I remembered it’s a Windows Server 2008 R2 permissions issue.  The resolution is quite simple and is often the solution when running into similar errors on Windows 7 and Windows Server 2008 R2.  In addition, I found the workaround documented in VMware KB 1025637.  Rather than launching the vCenter Server Linked Mode Configuration as you normally would by clicking on the icon, instead, right click the shortcut and choose Run as administrator.

SnagIt Capture 

You should find that launching the shortcut in the administrator context grants the installer the permissions necessary to complete Linked Mode configuration.

Windows 7 Launch Multiple Program Instances Shortcut

June 22nd, 2010

I don’t pretend to know all of the Windows keyboard shortcuts but I do maintain an arsenal of frequently used aka useful ones.  Here’s one that I discovered by accident which is helpful for applications which multiple instances can typically be spawned simultaneously.  Applications like the vSphere Client, PuTTY, Remote Desktop Connection, Command Prompt, maybe a web browser if you dislike browser tabs.

The shortcut:

With one instance of the desired application already launched (and visible on the Windows 7 taskbar), SHIFT + LEFT MOUSE CLICK on the application on the taskbar:

6-21-2010 10-05-36 PM

VIOLA!  An additional instance is spawned:

6-21-2010 10-06-36 PM

I’ve found immediate use for this with launching multiple vSphere Client instances.  Sure I have these frequently used applications pinned to my taskbar for one click launch efficiency but when the application already has one instance launched, the target to click on is ergonomically larger and thus easier to find.

This UI enhancement may also work with Vista.  I didn’t use that OS long enough to find out.  I’m not sure if Microsoft has an official name for this technology – surely there must be an acronym for it.  I’ll pay attention during the “Windows 7 was my idea” commercials as this was obviously someone’s idea and this trick could surface there.

ps. On the subject of Windows 7 enhancements.  While I do like and use the feature where an application is snapped to one of the four edges of the screen, at the same time I’ve developed a phobia about carefully navigating my mouse while dragging an application where I DO NOT want it to snap and take up a huge chunk of display real estate.  I’m passive aggressive particular about the dimensions of my application windows relative to everything else in the shared area.  The four edges of a Windows 7 display have tractor beams and when your mouse comes close to the edge, it sucks you the rest of the way in and before you know it, an app is maximized.  I’d bet *nix people don’t have these types of issues.

Active Directory Problems

June 13th, 2010

I’ll borrow an introduction from a blog post I wrote a few days ago titled NFS and Name Resolution because it pretty much applies to this blog post as well:

Sometimes I take things for granted. For instance, the health and integrity of the lab environment. Although it is “lab”, I do run some workloads which are key to keep online on a regular basis. Primarily the web server which this blog is served from, the email server which is where I do a lot of collaboration, and the Active Directory Domain Controllers/DNS Servers which provide the authentication mechanisms, mailbox access, external host name resolution to fetch resources on the internet, and internal host name resolution.

The workloads and infrastructure in my lab are 100% virtualized. The only “physical” items I have are type 1 hypervisor hosts, storage, and network. By this point I’ll assume most are familiar with the benefits of consolidation. The downside is that when the wheels come off in a highly consolidated environment, the impacts can be severe as they fan out and tip over down stream dependencies like dominos.

Due to my focus on VMware virtualization, the Microsoft Active Directory Domain Controllers hadn’t been getting the care and feeding they needed.  Quite honestly, there have several “lights out” situations in the lab due to one reason or another.  The lab infrastructure VMs and their underlying operating systems have taken quite a beating but continued running.  Occassionally a Windows VM would detect a need for a CHKDSK .  Similarly, Linux VMs wanted an FSCK.  But they would faithfully return to a login prompt.

A week ago today, the DCs succumbed to the long term abuse.  Symptoms were immediately apparent in that I could not connect to the Exchange 2010 server to access my email and calendar.  In addtion, I had lost access to the network drives on the file server.  Given the symptoms, I knew the issue was Active Diriectory related, however, I quickly found out the typcal short term remedies weren’t working.  I looked at the Event Logs for both DCs.  Both were a disaster and looking at the history, they had been ill for quite a long time.  I was going to have to really dig in to resolve this problem.

I spent several of the following evenings trying to resolve the problem.  As each day passed, anxiety was building because I was lacking email which is where I do a lot of work out of.  I had cleaned up AD meta data on both DCs, I had removed DCs to narrow the problem down, I examined DNS checking the integrity of AD integrated SRV records.  I had restored the DCs to an isolated network from prior backups to no avail.  Although AD was performing some base authentication, there were a handful of symptoms remaining which would indicate AD was still not happy.  A few of the big ones were:

  1. Exchange Services would either not start or would hang on starting
  2. SYSVOL and NETLOGON shares were not online on the DCs
  3. NETDIAG and DCDIAG tests on the DCs both had major failures, primarily inability to locate any DCs, Global Catalog Servers, time servers, or domain names

All of these problems utlimately tied to an error in the File Replication Service log on the DCs:

Event Type: Warning
Event Source: NtFrs
Event Category: None
Event ID: 13566
Date: 6/10/2010
Time: 9:15:56 PM
User: N/A
Computer: OBIWAN
File Replication Service is scanning the data in the system volume. Computer OBIWAN cannot become a domain controller until this process is complete. The system volume will then be shared as SYSVOL. 

To check for the SYSVOL share, at the command prompt, type:
net share 

When File Replication Service completes the scanning process, the SYSVOL share will appear.

The initialization of the system volume can take some time. The time is dependent on the amount of data in the system volume.

I had waited a long period of time for the scan to complete, but it had become apprent that the scan was never going to complete on its own.  After quite a bit of searching, I came up with Microsoft KB Article 263532 How to perform a disaster recovery restoration of Active Directory on a computer with a different hardware configuration.  Specifically, step 3j provided the answer to solving the root cause of the problem.  There is a registry value called BurFlags located in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NtFrs\Parameters\
Backup/Restore\Process at Startup\
.  The value needs to be set to d4 to allow SYSVOL to be shared out.

 Once this registry value was set, all of the problems I was experiencing went away. Exchange services started and I had access to my Email after a four day inbox vacation.  I had been through a few instances of AD meta data cleanup but this turned out to be a more complex problem than that.  I am thankful for internet search engines because I probably would have never solved this problem without the MS KB Article.  I was actually coming close to wiping my current AD and starting over, although I knew that would be pretty painful considering the integration of other components like Exchange, SQL, Certificate Services, DNS, Citrix, etc. that was tied to it.

New Microsoft .NET Framework Update Breaks vSphere Client

June 10th, 2010

Just a quick heads up to bring attention to an issue which I caught on Twitter.  VMware published KB 1022611 today which describes a new issue that is introduced by a recent Microsoft .NET Framework 2.0 SP2 & 3.5 SP1 update.  Upon installing the update, the vSphere Client stops working.  According to the article, the issue impacts ESX(i)3.5, 4.0, and vCenter 4.0.  Contrary to the topic of this blog post, I am not placing blame on Microsoft.  It remains unclear to me which company’s development staff is responsible for the software incompatibility.  Microsoft obviously issued the udpate which revealed the problem, but VMware has some skin in this as well in that they need to make sure they are following Microsoft .NET Framework development standards and best practices for their enterprise hypervisor management.

Key details from the VMware KB article:

The vSphere Clients, prior to the Update 1 release, cannot be used to access the vCenter Server or ESX hosts. A Microsoft update that targets the .NET Framework, released on June 9th 2010 is causing this issue. The update causes the vSphere Client to stop working.    To correct the issue there are two options that can be performed:

  • Remove the MS update from your Windows operating system. The vSphere Client works after the update is removed.

Note: This affects Windows XP, Windows 2003, Windows 2008, Windows Vista, and Windows 7.

VMware Workstation Upgrade to 7.1

May 26th, 2010

Microsoft Windows 7 Professional 64-bit
VMware Workstation 7.0.1 build-227600

I had heard VMware Workstation 7.1 was released.  Unfortunately, the VMware Workstation “check for updates” feature doesn’t seem to be serving its intended purpose as it told me no updates were available.

I downloaded the installation package manually and performed the upgrade.  Two reboots were required:

  1. After the uninstall of my previous version of Workstation
  2. After the install of Workstation 7.1

I hope the usability experience is better than my upgrade experience.  I realize some of the reboot business is on the Microsoft Windows 7 operating system but come on, would someone please figure this out?  Is there no way to perform an in place upgrade of Workstation to minimize the reboots to one?

What’s New in VMware Workstation 7.1

•Support for 8 virtual processors (or 8 virtual cores) and 2 TB virtual disks.

•Support for OpenGL 2.1 for Windows Vista and Windows 7 guests.

•Greatly improved DirectX 9.0 graphics performance for Windows Vista and Windows 7 guests. Up to 2x faster than Workstation 7.

•Launch virtualized applications directly from the Windows 7 taskbar to create a seamless experience between applications in your virtual machines and the desktop.

•Optimized performance for Intel’s Core i3, i5, i7 processor family for faster virtual machine encryption and decryption.

•Support for more Host and Guest Operating Systems, including: Hosts: Windows 2008 R2, Ubuntu 10.04, RHEL 5.4, and more Guests: Fedora 12, Ubuntu 10.04, RHEL 5.4, SEL 11 SP1, and more.

•Now includes built in Automatic Updates feature to check, download, and install VMware Workstation updates.

•Ability to import and export Open Virtualization Format (OVF 1.0) packaged virtual machines and upload directly to VMware vSphere, the industry’s best platform for building cloud infrastructures.

Windows 2008 R2 and Windows 7 on vSphere

March 28th, 2010

If you run Windows Server 2008 R2 or Windows 7 as a guest VM on vSphere, you may be aware that it was advised in VMware KB Article 1011709 that the SVGA driver should not be installed during VMware Tools installation.  If I recall correctly, this was due to a stability issue which was seen in specific, but not all, scenarios:

If you plan to use Windows 7 or Windows 2008 R2 as a guest operating system on ESX 4.0, do not use the SVGA drivers included with VMware Tools. Use the standard SVGA driver instead.

Since the SVGA driver is installed by default in a typical installation, it was necessary to perform a custom installation (or scripted perhaps) to exclude the SVGA driver for these guest OS types.  Alternatively, perform a typical VMware Tools installation and remove the SVGA driver from the Device Manager afterwards.  What you ended up with, of course, is a VM using the Microsoft Windows supplied SVGA driver and not the VMware Tools version shown in the first screenshot.  The Microsoft Windows supplied SVGA driver worked and provided stability as well, however one side effect was that mouse movement via VMware Remote Console felt a bit sluggish.

Beginning with ESX(i) 4.0 Update 1 (released 11/19/09), VMware changed the behavior and revised the above KB article in February, letting us know that they now package a new version of the SVGA driver in VMware Tools in which the bits are populated during a typical installation but not actually enabled:

The most effective solution is to update to ESX 4.0 Update 1, which provides a new WDDM driver that is installed with VMware Tools and is fully supported. After VMware Tools upgrade you can find it in C:\Program Files\Common Files\VMware\Drivers\wddm_video.

After a typical VMware Tools installation, you’ll still see a standard SVGA driver installed.  Following the KB article, head to Windows Device Manager and update the driver to the bits located in C:\Program Files\Common Files\VMware\Drivers\wddm_video:


The result is the new wddm driver, which ships with the newer version of VMware Tools, is installed: 

After a reboot, the crisp and precise mouse movement I’ve become accustomed to over the years with VMware has returned.  The bummer here is that while the appropriate VMware SVGA drivers get installed in previous versions of Windows guest operating systems, Windows Server 2008 R2 and Windows 7 require manual installation steps, much like VMware Tools installation on Linux guest VMs.  Add to this the fact that the automated installation/upgrade of VMware Tools via VMware Update Manager (VUM) does not enable the wddm driver.  In short, getting the appropriate wddm driver installed for many VMs will require manual intervention or scripting.  One thing you can do is to get the wddm driver installed in your Windows Server 2008 R2 and Windows 7 VM templates.  This will ensure VMs deployed from the templates have the wddm driver installed and enabled.

The wddm driver install method from VMware is helpful for the short term, however, it’s not the scalable and robust long term solution.  We need an automated solution from VMware to get the wddm driver installed.  It needs to be integrated with VUM.  I’m interested in finding out what happens with the next VMware Tools upgrade – will the wddm driver persist, or will the VMware Tools upgrade replace the wddm version with the standard version?  Stay tuned.

Update 11/6/10:  While working in the lab tonight, I noticed that with vSphere 4.1, the correct wddm video driver is installed as part of a standard VMware Tools installation on Windows 7 Ultimate x64 – no need to manually replace the Microsoft video driver with VMware’s wddm version as this is done automatically now.

Update 12/10/10: As a follow up to these tests, I wanted to see what happens when the wddm driver is installed under ESX(i) 4.0 Update 1 and its corresponding VMware Tools, and then the VM is moved to an ESX(i) 4.1 cluster and the VMware Tools are upgraded.  Does the wddm driver remain in tact, or will the 4.1 tools upgrade somehow change the driver?  During this test, I opted to use Windows 7 Ultimate 32-bit as the guest VM guinea pig.  A few discoveries were made, one of which was a surprise:

1.  Performing a standard installation of VMware Tools from ESXi 4.0 Update 1 on Windows 7 32-bit will automatically install the wddm driver, version as shown below.  No manual steps were required to install this driver forcing a 2nd reboot.  I wasn’t counting on this.  I expected the Standard VGA Graphics Adapter driver to be installed as seen previously.  This is good.

SnagIt Capture

After moving the VM to a 4.1 cluster and performing the VMware Tools upgrade, the wddm driver was left in tact, however, its version was upgraded to  This is also good in that the tools ugprade doesn’t negatively impact the desired results of leveraging the wddm driver for best graphics performance.

SnagIt Capture

More conclusive testing should be done with Windows 7 and Windows Server 2008 R2 64-bit to see if the results are the same.  I’ll save this for a future lab maybe.