Failed to connect to VMware Lookup Service

March 14th, 2014 by jason No comments »

Judging by the search results returned by Google, it looks like my blog is among the few virtualization blogs remaining which does not have a writeup on this topic.  It’s Friday so… why not.

Scenario:  vSphere 5.5 Update 1 VMware vSphere Web Client fails to log into vCenter Server (appliance version) with the following error returned:

Failed to connect to VMware Lookup Service

https://fqdn:7444/lookupservice/sdk –

SSL certificate verification failed.

Snagit Capture

Contributing factors in my case which may have played a role in this once working environment:

  1. Recently upgraded vCenter 5.5.0 Server appliance to Update 1 (unlikely as other similar environments were not impacted after upgrade)
  2. This particular vCenter appliance was deployed as a vApp from a vCloud Director catalog (likely  but unknown at this time if a customization was possible or attempted during deployment)
  3. The hostname of the appliance may have been changed recently (very likely)

The solution is quite simple.

  1. Log into the vCenter Server appliance management interface (https://fqdn:5480/)
  2. Navigate to the Admin tab
  3. Certificate regeneration enabled: choose Yes
  4. Click the Submit button
  5. Navigate to the System tab
  6. Reboot the appliance

After the appliance reboots

  1. Log into the vCenter Server appliance management interface (https://fqdn:5480/)
  2. Navigate to the Admin tab
  3. Certificate regeneration enabled: choose No
  4. Click the Submit button
  5. Log out of the vCenter Server appliance management interface
  6. Log into the VMware vSphere Web Client normally

Admittedly I recalled the Certificate regeneration feature first by logging into the vCenter Server appliance management interface, but then verified with a search to ensure the purpose of the Certificate regeneration feature.  The search results turned up Failed to connect to VMware Lookup Service – SSL Certificate Verification Failed (among many other blog posts as mentioned earlier) in addition to VMware KB 20333338 Troubleshooting the vCenter Server Appliance with Single Sign-On login.  Both more or less highlight a discrepancy between the appliance hostname and the SSL certificate resulting in the need to regenerate the certificate to match the currently assigned hostname.

I ran across another issue this week during the Update 1 upgrade to the vCenter appliance which I may or may not get to writing about today.

At any rate, have wonderful and Software Defined weekend!

VMware Releases vSphere PowerCLI 5.5 R2

March 12th, 2014 by jason No comments »

I stumbled across some interesting news shared by Alan Renouf on Facebook this morning – an R2 release of vSphere PowerCLI 5.5 (Build 1649237).  New in R2 per the release notes:

  • Access to the vCenter Server SRM public API (Connect-SRMServer and Disconnect-SRMServer cmdlets) – an exciting addition for sure
  • Support for adding and removing tags and tag categories found in the next generation vSphere web client
  • Configuration and reporting of EVC mode for vSphere clusters
  • Management of security policies for the vSS and its portgroups
  • New support for MS Windows PowerShell 4.0
  • Support for vSphere hosts configured for IPv6
  • Added migration priority support for vMotion (VMotionPriority parameter in conjunctionw ith the Move-VM cmdlet)
  • Get-Datastore cmdlet
    • RelatedObject paremeter extended to accept the Harddisk object
    • now allows filtering by cluster
  • Enhanced Get-Stat and Get-StatType cmdlets
  • Support added for e1000e vNICs
  • All values for DiskStorageFormat can be specified during VM cloning operations
  • 64-bit mode support for New-OSCustomizationSpec and Set-OSCustomizationSpec cmdlets
  • ToolsVersion property added to VMGuest which returns a string
  • Get-VirtualSwitch and Get-DVSwitch cmdlets support virtual port groups as a RelatedObject
  • Get-VM cmdlet enhanced to retrieve a list of VMs by virtual switch
  • Miscellaneous bug fixes

VMware vSphere PowerCLI 5.5 R2 supports vSphere 4.1 through vSphere 5.5 as well as Microsoft Windows PowerShell versions 2.0, 3.0, and new in R2 4.0.

Thank you Alan and thank you VMware!

VMTurbo’s Disruptive Software-Driven Control Expands Across Storage and Fabric To Realize Full Value of Virtualization

February 9th, 2014 by jason No comments »

Press Release

VMTurbo’s Disruptive Software-Driven Control Expands Across Storage and Fabric To Realize Full Value of Virtualization

VMTurbo Operations Manager Enables Customers to Realize 30%Improvement in Utilization while assuring application workload performance

BOSTON, MA – January 27, 2014 – VMTurbo, provider of the only Software-Driven Control for virtualized environments, today announced a new version of its flagship product, VMTurbo Operations Manager, enhanced with control modules for storage and fabric to drive virtualized environments to their desired state and maintain control in that state across the data center and IT stack.  These new solutions enable 30%  improvement in utilization while providing greater control over all aspects of the environment the application workload touches – from compute and storage to fabric and cloud. 

One of the major advancements in this release is management of the Converged Fabric layer with Cisco (CSCO) UCS support. Not only does VMTurbo provide unprecedented visibility into UCS from the fabric interconnect down to individual blades, it also enables control of UCS to manage real demand for UCS ports to maximize port utilization and avoid unnecessary port licensing costs.

“We’ve made a significant investment in UCS and are happy with it but it’s a challenge to manage,” said Jonathan Brown, Desktop Administrator at Beaufort Memorial Hospital (www.bmhsc.org). “VMTurbo is the only solution we’ve found on the market that helps us understand the inner workings of UCS so we can better manage it. We love VMTurbo, and are excited for all the new features to help us manage the future growth of our environment.”

VMTurbo is also disrupting enterprise software with its model of “easy to try, buy, deploy, and use”. Customers download VMTurbo Operations Manager and realize value instantly – unlike traditional management software which takes several months to install and perform after significant integration costs.  In fact, VMTurbo offers customers a free health check assessment of their virtual environments.  With VMTurbo, customers can break free from expensive, monitoring solutions that fail to eliminate reactive and labor-intensive IT fire fighting..  

“I learned more about my data center in 15 minutes with VMTurbo than I did in the last five years,” said Chuck Green, CIO of AlphaMaxx Healthcare, Inc., the premiere NCQA-accredited perinatal population health management firm.  “It’s truly a paradigm shift.”

90% of customers that have implemented VMTurbo’s Software-Driven control system to manage their virtualized data centers and cloud infrastructures report a return-on-investment of less than three months from purchase – an unparalleled breakthrough disrupting traditional enterprise management software (Source: TechValidate).

VMTurbo was recently recognized last week by Forbes as one of 2014 America’s Most Promising Companies.  Earlier in the year VMTurbo received the JP Morgan Hall of Innovation award, being named one of the most innovative technologies in the data center.

“VMTurbo’s technology is helping JPMorgan Chase optimize the utilization of virtual environments and thereby supporting a move from reactive to predictive workload management,” said George Sherman, Head of Compute Services at JPMorgan Chase. “Automation will enable our support teams to focus on higher value activity by preventing incidents and dynamically optimizing virtual environments.” 

VMTurbo Operations Manager

VMTurbo Operations Manager is the only product on the market understanding application workload performance, resource utilization and constraints in virtualized datacenter and cloud deployments to drive an organization’s environment to its desired state – that state of perpetual health where application performance is assured while maximizing efficiency – while providing control over all aspects of the environment the application workload touches, from compute and storage to fabric and cloud. While competitive solutions focus on viewing – monitoring systems to send alerts requiring operational staff to troubleshoot and remedy issues – VMTurbo Operations Manager ties the viewing with the doing, so IT Operations staff can elevate their focus from reactive to strategic. To try VMTurbo Operations Manager in your own environment, visit www.vmturbo.com/download or for a free health check assessment, call 1.877.978.8818 .

For more detailed information on VMTurbo Operations Manager, visit vmturbo.com/operations-manager.

VMTurbo Storage Control Module

VMTurbo’s Storage Control Module ensures applications get the storage performance they require to operate reliably while enabling efficient use of storage infrastructure – thus preventing unnecessary over provisioning.  This module helps users solve their pressing storage performance and cost challenges, maximize their existing storage investments and embrace the adoption of advanced features and packaging such as NetApp Clustered Data ONTAP (cluster mode) and FlexPod. For more detailed information on VMTurbo Storage Control Module, visit www.vmturbo.com/storage-resource-management.

VMTurbo Fabric Control Module

Modern compute platforms and blade servers have morphed to fabrics unifying compute, network, virtualization and storage access into a single integrated architecture.  Furthermore, fabrics like Cisco (CSCO) UCS form the foundation of a programmable infrastructure for today’s private clouds and virtualized datacenters, the backbone of converged infrastructure offerings from VCE vBlock and NetApp FlexPod. 

With the addition of this Fabric Control Module, VMTurbo’s software-driven control system ensures workloads get the compute and network resources they need to perform reliably while maximizing the utilization of underlying blades and ports. For more detailed information on VMTurbo Fabric Control Module, visit  www.vmturbo.com/ucs-management 

About VMTurbo

VMTurbo’s Software-Driven Control platform enables organizations to manage cloud and enterprise virtualization environments to maximize infrastructure investments while assuring application performance. VMTurbo’s patent-pending Economic Scheduling Engine dynamically adjusts configuration, resource allocation and workload placement to meet service levels and business goals, and is the only technology capable of closing the loop in IT operation by automating the decision-making process to maintain an environment in its desired state. The VMTurbo platform first launched in August 2010 and since that time more than 10,000 cloud service providers and enterprises worldwide have deployed the platform, including JP Morgan Chase, Colgate-Palmolive and Ingram Micro. Using VMTurbo, our customers ensure that applications get the resources they need to operate reliably, while utilizing their most valuable infrastructure and human resources most efficiently. For more information, visit www.vmturbo.com.

vCenter Server Appliance 5.5 root account locked out after password expiration

January 10th, 2014 by jason 1 comment »

Thanks to Chris Colotti, I learned of a new VMware KB article today which could potentially have wide spread impact, particularly in lab, development, or proof of concept environments.  The VMware KB article number is 2069041 and it is titled The vCenter Server Appliance 5.5 root account locked out after password expiration.

In summary, the root account of the vCenter Server Appliance version 5.5 becomes locked out 90 days after deployment or root account password change.  This behavior is by design which follows a security best practice of password rotation.  In this case, the required password rotation interval is 90 days after which the account will be forcefully locked out if not changed.

The KB article describes processes to prevent a forced lockout as well as unlocking a locked out root account.

Approximately 90 days have elapsed since the release of vSphere 5.5 and I imagine this issue will quickly begin surfacing in large numbers where the vCenter Server Appliance 5.5 has been deployed using system defaults.

Update 6/16/16: For more information on vCenter Server Appliance password policies, including the local root account, check out vCSA 6.0 tricks: shell access, password expiration and certificate warnings.

vSphere Consulting Opportunity in Twin Cities

December 14th, 2013 by jason No comments »

If you know me well, you know the area I call home.  If you’re a local friend, acquaintance, or member any of the three Minnesota VMware User Groups, then I have an opportunity that has crossed my desk which you or someone you know may be interested in.

A local business here in the Twin Cities has purchased vSphere and EMC VNXe storage infrastructure and is looking for a Consulting Engineer to deploy the infrastructure per an existing design.

Details:

  • Install and configure VMware vSphere 5.1 on two hosts
  • Install and configure VMware vCenter
  • Install and configure VMware Update Manager
  • Configure vSphere networking
  • Configure EMC VNXe storage per final design.

It’s a great opportunity to help a locally owned business deploy a vSphere infrastructure and I would think this would be in the wheelhouse of 2,000+ people I’ve met while running the Minneapolis VMware User Group.  As much as I’d love to knock this out myself, I’m a Dell Storage employee and as such I’m removing myself as a candidate for the role.  The best way I can help is to get the word out into the community.

If you’re interested, email me with your contact information and I’ll get you connected to the Director.

Happy Holidays!

Storage Center 5.6 Released

November 25th, 2013 by jason No comments »

I don’t have the latest and greatest Dell Compellent SC8000 controllers or SC220 2.5″ drive enclosures in my home lab although I dream nightly about Santa unloading some on me this Christmas.  What I do have is an older Series 20 and I am thankful for that.  But having an older storage array doesn’t mean I cannot leverage some of the latest and greatest features and operating systems available for datacenters.

Storage Center 5.6 was released just a short time ago and it ushers in some feature and platform support currently built into Storage Center 6.x as well as a large number of bug fixes.  This is a big win for me and anyone with 32-bit system (Series 30 or below) needing these features because SCOS 6.x is 64-bit only for Series 40 and newer which today includes the SC8000.

So what are these new features in 5.6 and why am I so excited?  I’m glad you asked.  For this guy, and on top of the list, it’s full support of all VAAI primitives.  Storage Center 5.5 and older boasted support of the block zeroing primitive.  Space Reclamation was there as well although that primitve alone did not satisfy the other component of the thin provisioning primitive which was STUN.

Shown below a Storage Center 5.5 datastore where I lack Atomic Test and Set (aka Hardware Assisted Locking) and XCOPY.  I have block zeroing and Space Reclamation using the Free Space Recovery agent for vSphere guest VMs using physical RDMs. VAAI support status can be obtained in full using esxcli:

Snagit Capture

Or in part using the vSphere Client GUI:

Snagit Capture

After the Storage Center 5.6 upgrade, I’ve got additional VAAI primitive support where Clone in most cases is going to be the biggest one in terms of fabric and host efficiency and performance. Not shown is support for Thin Provisioning Stun but that has been added as well:

Snagit Capture

The vSphere Client GUI now reflects full VAAI support after the 5.6 upgrade:

Snagit Capture

What else? Added support for vSphere 5.5 as an operating system type:

Snagit Capture

Last but not least, added support for Windows 2012 and some of its features including Offloaded Data Transfer, Thin Provisioning, Space Reclamation, and Server Objects:

Snagit Capture

Storage Center 5.6 also adds new storage features which are storage host agnostic such as Background Media Scans (BMS) as well as improved disk and HBA management for server objects.  And the bug fixes I mentioned earlier – refer to the SCOS 5.6 Release Notes for details.

To wrap this up, if you’ve got an older Storage Center model and you want support for these new features while avoiding a forklift upgrade, Storage Center Operating System 5.6 is the way to go.

Microsoft Sysprep Change in vCloud Director 5.5

November 18th, 2013 by jason No comments »

If you’re like me, you still support legacy Windows operating systems from time to time.  Let’s face it, Windows Server 2003 was a great server operating system and will probably remain in some environments for quite a while.  I won’t at all be surprised if the Windows Server 2003 legacy outlasts that of Windows XP.  To that point, even the VCAP5-DCA  exam I sat a few weeks ago used Windows Server 2003 guests in the lab.

All of that being said in what is almost the year 2014, hopefully you are not still deploying Windows Server 2003 as a platform to deliver applications and services in a production environment.  However, if you are and you’re using VMware vCloud Director 5.5, you should be aware of subtle changes which I noticed while reading through the documentation.  Page 31 of the vCloud Director 5.5 Installation and Upgrade Guide to be exact.

In previous versions of vCloud Director including 5.1, Microsoft Sysprep files were placed in a temporary directory within operating system specific folders on the first cloud cell server in the cluster.  The next step was to invoke the /opt/vmware/vcloud-director/deploymentPackageCreator/createSysprepPackage.sh script which bundled all of the Sysprep files into a /opt/vmware/vcloud-director/guestcustomization/windows_deployment_package_sysprep.cab file.  At this point, Sysprep was installed and configured on the first cell server.  It could then optionally be distributed by way of copying the .cab file and the vcloud_sysprep.properties file to the guestcustomization directory of the other cell servers in the cluster.  I call this step optional because not all vCloud deployments will have multiple cell servers.  If multiple cell servers did exist, you’d likely want all of them to be able to perform guest customization duties for legacy Windows operating systems in the catalog and thus this optional step would be required.

So a few things have changed now in 5.5.  First, the Windows operating system specific folder names have changed to match the folder names which vCenter Server has always used historically (see VMware KB 1005593) and on this note, Windows 2000 Server support has been put out to pasture in vCD 5.5.

Version pre-vCD 5.5 vCD 5.5
Windows 2000 /win2000 unsupported
Windows Server 2003 (32-bit) /win2k3 /svr2003
Windows Server 2003 (64-bit) /win2k3_64 /svr2003-64
Windows XP (32-bit) /winxp /xp
Windows XP (64-bit) /winxp_64 /xp-64

Next, the method to create the Sysprep package and distribute it to the other cell servers has changed.  The createSysprepPackage.sh script no longer exists and as a result, a bundled .cab file is not created.  Instead, the Sysprep files are copied in their entirety to their new directory names within the directory /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep.  So what you need to do here is create the directory structure under $VCLOUD_HOME and SCP the Sysprep files to each of the cell servers.  I’ve provided the directory creation commands below:

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/svr2003

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/sv42003-64

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/xp

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/xp-64

As the documentation reminds us, the Sysprep files must be readable by the user vcloud.vcloud (this user is created on each cell server during the initial vCloud Director installation) and that can be ensured by running the following command:

chown -R vcloud.vcloud $VCLOUD_HOME/guestcustomization

These installation changes are important to note if you’re deploying a net new vCloud Director 5.5 environment and there is a need for legacy Windows OS vAPP guest customization.  A vCloud Director 5.5 upgrade from previous versions will perform the necessary Sysprep migration steps automatically.  Lastly, Sysprep won’t be needed in vCloud environments where guest customization isn’t required or legacy versions of Windows aren’t being deployed and customized (Beginning with Windows Vista and Windows Server 2008, Sysprep is bundled within the operating system).