One of the interesting aspects of shared infrastructure is stumbling across configuration changes made by others who share responsibility in managing the shared environment. This is often the case in the lab but I’ve also seen it in every production environment I’ve supported to date as well. I’m not pointing any fingers – my back yard is by no means immaculate. Moreover, this bit is regarding automation, not placing blame (Note the former is productive while the latter is not).
Case in point this evening when I was attempting to perform a simple remediation of a vSphere 5.1 four-host cluster via Update Manager. I verified the patches and cluster configuration, hit the remediate button in VUM, and left the office. VUM, DRS, and vMotion does the heavy lifting. I’ve done it a thousand times or more in the past in environments 100x this size.
I wrap up my 5pm appointment on the way home from the office, have dinner with the family, and VPN into the network to verify all the work was done. Except nothing had been accomplished. Remediation on the cluster was a failure. Looking at the VUM logs reveals 75% of the hosts being remediated contain virtual machines with attached devices preventing VUM, DRS, and vMotion from carrying out the remediation.
Obviously I know how to solve this problem but to manually check and strip every VM of it’s offending device is going to take way too long. I know what I’m supposed to do here. I can hear the voices in my head of PowerShell gurus Alan, Luc, etc. saying over and over the well known automation battle cry “anything repeated more than once should be scripted!”
That’s all well and good, I completely get it, but I’m in that all too familiar place of:
- Carrying out the manual tasks will take 30 minutes.
- Authoring, finding, testing a suitable PowerShell/PowerCLI script to automate will also take 30 minutes, probably more.
- FML, I didn’t budget time for either of the above.
There is a middle ground. I view it as drive-through efficiency automation. It’s call PowerGUI and it has been around almost forever. In fact, it comes from Quest which my employer now owns. And I’ve already got it along with the PowerPacks and Plug-ins installed on my new Dell Precision M4800 laptop. Establishing a PowerGUI session and authenticating with my current infrastructure couldn’t be easier. From the legacy vSphere Client, choose the Plug-ins pull down, PowerGUI Administrative Console.
The VMware vSphere Management PowerPack ships stock with not only the VM query to find all VMs with offensive devices attached, but also a method to highlight all the VMs and Disconnect.
Depending on the type of device connect to the virtual machines, VUM may also be able to handle the issue as it has the native ability to Disable any removable media devices connect to the virtual machines on the host. In this case, the problem is solved with automation (I won’t get beat up on Twitter) and free community (now Dell) automation tools. Remediation completed.
RVTools (current version 3.6) also has identical functionality to quickly locate and disconnect various devices across a virtual datacenter. Click on the image below to read more about RVTools.
Click on the image below to read more about PowerGUI.
An equally effective method for solving this quandry is to utilize an option within the Update Manager remediation windows to do the work for you.
http://i.imgur.com/RpNvPut.jpg