Posts Tagged ‘vCenter Server’

vCenter Server Appliance 5.5 root account locked out after password expiration

January 10th, 2014

Thanks to Chris Colotti, I learned of a new VMware KB article today which could potentially have wide spread impact, particularly in lab, development, or proof of concept environments.  The VMware KB article number is 2069041 and it is titled The vCenter Server Appliance 5.5 root account locked out after password expiration.

In summary, the root account of the vCenter Server Appliance version 5.5 becomes locked out 90 days after deployment or root account password change.  This behavior is by design which follows a security best practice of password rotation.  In this case, the required password rotation interval is 90 days after which the account will be forcefully locked out if not changed.

The KB article describes processes to prevent a forced lockout as well as unlocking a locked out root account.

Approximately 90 days have elapsed since the release of vSphere 5.5 and I imagine this issue will quickly begin surfacing in large numbers where the vCenter Server Appliance 5.5 has been deployed using system defaults.

Update 6/16/16: For more information on vCenter Server Appliance password policies, including the local root account, check out vCSA 6.0 tricks: shell access, password expiration and certificate warnings.

Single Sign-On Warning 25000

November 12th, 2013

Up to this point, I’ve deployed several net new instances of vCenter Server 5.5 and of course its essential components including Single Sign-On, Inventory Service, next generation Web Client, and the legacy vSphere Client.  Most of these deployments leveraged the vCenter appliance.  Using the appliance is a very easy to deploy vCenter because all of the essential components are pre-installed in the appliance and only need to be configured.

One area I hadn’t tackled much yet are upgrades of existing Windows-based vCenter environments to vSphere 5.5.  Having recently completed an inline upgrade of vCloud Director 5.1.2 to 5.5, it was now time to upgrade said vCloud’s underlying vSphere 5.1 (Update 1a I believe) virtual infrastructure.   Prior to starting the upgrade, I took the necessary precautions of getting a point in time snapshot of the vCenter Server, the vCloud Director Cells, and the Microsoft SQL Server databases for each (three total: SSO, vCenter, and vCD).  I accomplished this using array based snapshots – in this case Dell Compellent Storage Center Replays.

I launched autorun from the vCenter 5.5 installation media.  I opted for the custom installation and started with the Single Sign-On (SSO) upgrade from 5.1 to 5.5.  During the installation, I was met with

Warning 25000.  Please verify that the SSL certificate for your vCenter Single Sign-On 5.1 SSL is not expired.  If it did expire, please replace it with a valid certificate before upgrading to vCenter Single Sign-On 5.5.

Snagit Capture

In this particular environment, self-signed certificates from VMware were in use.  I know that this environment was deployed new less than two years ago and a verification of the SSL certificates in use proved that none were expired.  But because SSO and vCenter are such integral components to vCloud Director, I didn’t want to proceed without further vetting this out.

Google.

Upgrade from vSphere 5.1 to vSphere 5.5 rolls back after importing Lookup Service data (2060511) – This KB article describes a situation in which Warning 25000 results when a registry value on the existing Windows-based SSO 5.1 server does not match a field on the SSL certificate.  The resolution involves simply changing the registry value to match that which is on the SSL certificate.  I won’t repeat the details because you can read the KB article yourself.  Furthermore it didn’t resolve the problem in this case because the field on my SSL certificate and the registry key were an identical match.

Upgrading to VMware vCenter Single Sign-On 5.5 displays the error: Warning 25000 (2061478) – This KB article describes a problem for which there is no resolution. However, there is a workaround and that involves changing service_id and service.properties files.  More detail is available in the KB article and again the symptoms in the log files weren’t a close match.

The Trouble With SSL Certificates and Upgrading to VMware SSO 5.5 – Then I took a look at Michael Webster’s blog article on precisely the same error message.  Michael briefly discusses the two SSL certificate deployment models and then digs into VMware KB 2060511 mentioned above.  While the information in Michael’s blog article reassured me I was not alone in my journey, KB 2060511 didn’t solve my problem either.  But sometimes the value of blog articles is not only in the original author’s content, but also in the follow up comments from the readers.  Such was the case here.  A number of Michael’s readers responded by saying they were essentially in the same boat I’m in – it sounds like KB 2060511, but in the end this article doesn’t have the solution because there was nothing wrong with their SSO registry values.  The readers found no choice but to push onward beyond Warning 25000 with fingers crossed.  As it turned out in my as well as with some others, Warning 25000 was benign in nature and the installation completed successfully with no rollback.

In summary, this blog post does not represent global authority to ignore Warning 25000.  Rather it is meant to highlight one particular scenario where Warning 25000 may present itself and the actions that were taken to work through the problem.  I can’t stress enough the importance of the SSO component of vCenter going forward.  If any conclusion can be drawn here, it is that a backup of the infrastructure components should be secured before committing to the upgrade steps.  In this case, snapshots are the quickest and easiest method to provide data protection and recovery.  Although vSphere snapshots would work in some deployment architectures, recovering an environment when the environment being upgraded is managing the snapshots could be a challenge.  That is why I chose an out of band array based snapshot in this instance.

I would also like to point out in closing that vSphere 5.5 is still relatively new and VMware appears to still be chasing down all possible causes, resolutions, and workarounds to Warning 25000.  New information as well as VMware KB articles may develop subsequent to this writing so it may be worth continuing your own Google searching beyond this point.

Have a great week!

A Look At vCenter 5.5 SSO RC Installation

August 30th, 2013

This week at VMworld 2013, I attended a few sessions directly related to vCenter 5.5 as well as its components, one of which is vCenter Single Sign On (SSO):

  • VSVC5234 – Extreme Performance Series: vCenter of the Universe
  • VSVC4830 – vCenter Deep Dive

First of all, both sessions were excellent and I highly recommend viewing them if you have access to the post conference recordings. 

If you followed my session tweets or if perhaps you’ve read half a dozen or more already available blog posts on the subject, you know that several improvements have been made to vCenter SSO for the vSphere 5.5 release.  For instance:

  • Completely re-written from the ground
  • Multi-master architecture
  • Native replication mechanism
  • SSO now has site awareness (think of the possibilities for HA stretched clusters)
  • MMC based diagnostic suite available as a separately maintained download
  • The external database and its preparation dependency has been removed
  • Database patitioning to improve both scalability and performance (this was actually added in 5.1 but I wanted to call it out)
  • Revamped multi-site deployment architecture
  • Full Mac OS X web client support including remote console
  • Improved certificate management
  • Multi-tenant capabilities
  • Drag ‘n’ Drop in the 5.5 web client

With some of the new features now identified and VMware’s blessing, have a look at the installation screens and see if you can spot the differences as compared to a vCenter 5.1 SSO installation.  These stem from a manual installation of SSO, not an automated installation of all vCenter components (by the way, the next gen web client is now installed as part of an automated vCenter 5.5 installation whereas it was not in 5.1).  Keep in mind these were pulled from a release candidate version and may change when vCenter 5.5 GAs at a future date.

I noticed one subtle change here – clicking on the Microsoft .NET 3.5 SP1 link in Windows 2008R2 actually installs the feature rather than just throwing up a dialogue box asking you to install the feature yourself.

Snagit Capture

As this is a manual installation, we have the option to use the default or specify the installation location.  Best practice is to install all vCenter components together so that they can communicate at server bus speed and won’t be impacted by network latency.  However, for larger scale environments, SSO should be isolated on a separate server with five or more vCenter Servers in the environment.  On a somewhat related note, the Inventory Service may benefit from an installation on SSD, again in large infrastructures.

Snagit Capture

We won’t likely see this in the GA version.

Snagit Capture

We’re going through the process of installing vCenter version 5.5 but in terms of the SSO component, again this is a complete re-write and bears the respective version of 2.0.

Snagit Capture

We always read the EULA in full and agree to the license terms and conditions.

Snagit Capture

 

Snagit Capture

Big changes here.  Note the differences in the deployment models compared to the previous 5.1 version – previous deployment models are honored through an upgrade to 5.5.  Again, this is where the VMworld sessions noted above really go into detail. 

Snagit Capture

the System-Domain namespace has been replaced with vsphere.local.

Snagit Capture

The new site awareness begins here.

Snagit Capture

Snagit Capture

Snagit Capture

Snagit Capture

I hope you agree that SSO installation in vCenter 5.5 has been simplified while many new features have been added at the same time.

As always, thank you for reading and it was a pleasure to meet and see everyone again this year at VMworld.

 

vSphere 5.1 Update 1 Update Sequence

May 6th, 2013

Not so long ago, VMware product releases were staggered.  Major versions of vSphere would launch at or shortly after VMworld in the fall, and all other products such as SRM, View, vCloud Director, etc. would rev on some other random schedule.  This was extremely frustrating for a vEvangelist because we wanted to be on the latest and greatest platform but lack of compatibility with the remaining bolt-on products held us back.

While this was a wet blanket for eager lab rats, it was a major complexity for production environments.  VMware understood this issue and at or around the vSphere 5.0 launch (someone correct me if I’m wrong here), all the development teams in Palo Alto synchronized their watches & revd product in essence at the same time.  This was great and it added the much needed flexibility for production environment migrations.  However, in a way it masked an issue which didn’t really exist before by virtue of product release staggering – a clear and understandable order of product upgrades.  That is why in March of 2012, I looked at all the product compatibility matrices and sort of came up with my own “cheat sheet” of product compatibility which would lend itself to an easy to follow upgrade path, at least for the components I had in my lab environment.

vSphere 5.1 Update 1 launched on 4/25/13 and along with it a number of other products were revd for compatibility.  To guide us on the strategic planning and tactical deployment of the new software bundles, VMware issued KB Article 2037630 Update sequence for vSphere 5.1 Update 1 and its compatible VMware products.

Snagit Capture

Not only does VMware provide the update sequencing information, but there are also exists a complete set of links to specific product upgrade procedures and release notes which can be extremely useful for planning and troubleshooting.

The vCloud Suite continues to evolve providing agile and elastic infrastructure services for businesses around the globe in a way which makes IT easier and more practical for consumers but quite a bit more complex on the back end for those who must design, implement, and support it.  Visit the KB Article and give it 5 stars.  Let VMware know this is an extremely helpful type of collateral for those in the trenches.

Book Review: VMware vSphere 5 Building a Virtual Datacenter

March 4th, 2013

Snagit Capture

Publication Date: August 30, 2012 | ISBN-10: 0321832213 | ISBN-13: 978-0321832214 | Edition: 1

I’m long overdue on book reviews and I need to start off with an apology to the authors for getting this one out so late.  The title is VMware vSphere 5 Building a Virtual Datacenter by Eric Maillé and René-François Mennecier (Foreword by Chad Sakac and Technical Editor Tom Keegan).  This is a book which caught me off guard a little because I was unaware of the authors (both in virtualization and cloud gigs at EMC Corporation) but nonetheless meeting new friends in virtualization is always pleasant surprise.  It was written prior to and released at the beginning of September 2012 with vSphere coverage up to version 5.0 which launched early in September 2011.

The book starts off with the first two chapters more or less providing a history of VMware virtualization plus coverage of most of the products and where they fit.  I’ve been working with VMware products since just about the beginning and as such I’ve been fortunate to be able to absorb all of the new technology in iterations as it came over a period of many years.  Summarizing it all in 55 pages felt somewhat overwhelming (this is not by any means a negative critique of the authors’ writing).  Whereas advanced datacenter virtualization was once just a concatenation of vCenter and ESX, the portfolio has literally exploded to a point where design, implementation, and management has gotten fairly complex for IT when juggling all of the parts together.  I sympathize a bit for late adopters – it really must feel like a fire hose of details to sort through to flesh out a final bill of materials which fits their environment.

From there, the authors move on to cover key areas of the virtualized and consolidated datacenter including storage and networking as well as cluster features, backup and disaster recovery (including SRM), and installation methods.  In the eighth and final chapter, a case study is looked at in which the second phase of a datacenter consolidation project must be delivered.  Last but not least is a final section titled Common Acronyms which I’ll unofficially call Chapter 9.  It summarizes and translates acronyms used throughout the book.  I’m not sure if it’s unique but it’s certainly not a bad idea.

To summarize, the book is 286 pages in length, not including the index.  It’s not a technical deepdive which covers everything in the greatest of detail but I do view it as a good starting point which is going to answer a lot of questions for beginners and beyond as well as provide some early guidance along the path of virtualization with vSphere.  The links above will take you directly to the book on Amazon where you can purchase a paperback copy or Kindle version of the book.  Enjoy and thank you Eric and René-François.

Chapter List

  1. From Server Virtualization to Cloud Computing
  2. The Evolution of vSphere 5 and its Architectural Components
  3. Storage in vSphere 5
  4. Servers and Network
  5. High Availability and Disaster Recovery Plan
  6. Backups in vSphere 5
  7. Implementing vSphere 5
  8. Managing a Virtualization Project
  9. Common Acronyms

VMworld 2012 Announcements – Part I

August 27th, 2012

VMworld 2012 is underway in San Francisco.  Once again, a record number of attendees is expected to gather at the Moscone Center to see what VMware and their partners are announcing.  From a VMware perspective, there is plenty.

Given the sheer quantity of announcements, I’m actually going to break up them up into a few parts, this post being Part I.  Let’s start with the release of vSphere 5.1 and some of its notable features.

Enhanced vMotion – the ability to now perform a vMotion as well as a Storage vMotion simultaneously. In addition, this becomes an enabler to perform vMotion without the shared storage requirement.  Enhanced vMotion means we are able to migrate a virtual machine stored on local host storage, to shared storage, and then to local storage again.  Or perhaps migrate virtual machines from one host to another with each having their own locally attached storage only.  Updated 9/5/12 The phrase “Enhanced vMotion” should be correctly read as “vMotion that has been enhanced”.  “Enhanced vMotion” is not an actual feature, product, or separate license.  It is an improvement over the previous vMotion technology and included wherever vMotion is bundled.

Snagit Capture

Enhanced vMotion Requirements:

  • Hosts must be managed by same vCenter Server
  • Hosts must be part of same Datacenter
  • Hosts must be on the same layer-2 network (and same switch if VDS is used)

Operational Considerations:

  • Enhanced vMotion is a manual process
  • DRS and SDRS automation do not leverage enhanced vMotion
  • Max of two (2) concurrent Enhanced vMotions per host
  • Enhanced vMotions count against concurrent limitations for both vMotion and Storage vMotion
  • Enhanced vMotion will leverage multi-NIC when available

Next Generation vSphere Client a.k.a. vSphere Web Client – An enhanced version of the vSphere Web Client which has already been available in vSphere 5.0.  As of vSphere 5.1, the vSphere Web Client becomes the defacto standard client for managing the vSphere virtualized datacenter.  Going forward, single sign-on infrastructure management will converge into a unified interface which any administrator can appreciate.  vSphere 5.1 will be the last platform to include the legacy vSphere client. Although you may use this client day to day while gradually easing into the Web Client, understand that all future development from VMware and its partners now go into the Web Client. Plug-ins currently used today will generally still function with the legacy client (with support from their respective vendors) but they’ll need to be completely re-written vCenter Server side for the Web Client.  Aside from the unified interface, the architecture of the Web Client has scaling advantages as well.  As VMware adds bolt-on application functionality to the client, VMware partners will now have the ability to to bring their own custom objects objects into the Web Client thereby extending that single pane of glass management to other integrations in the ecosystem.

 

Here is a look at that vSphere Web Client architecture:

Snagit Capture

Requirements:

  • Internet Explorer / FireFox / Chrome
  • others (Safari, etc.) are possible, but will lack VM console access

A look at the vSphere Web Client interface and its key management areas:

Snagit Capture

Where the legacy vSphere Client fall short and now the vSphere Web Client solves these issues:

  • Single Platform Support (Windows)
    • vSphere Web Client is Platform Agnostic
  • Scalability Limits
    • Built to handle thousands of objects
  • White Screen of Death
    • Performance
  • Inconsistent look and feel across VMware solutions
    • Extensibility
  • Workflow Lock
    • Pause current task and continue later right where you left off (this one is cool!)
    • Browser Behavior
  • Upgrades
    • Upgrade a Single serverside component

 vCloud Director 5.1

In the recent past, VMware aligned common application and platform releases to ease issues that commonly occurred with compatibility.  vCloud Director, the cornerstone of the vCloud Suite, is obviously the cornerstone in how VMware will deliver infrastructure, applications, and *aaS now and into the future. So what’s new in vCloud Director 5.1?  First an overview of the vCloud Suite:

Snagit Capture

And a detailed list of new features:

  • Elastic Virtual Datacenters – Provider vDCs can span clusters leveraging VXLAN allowing the distribution and mobility of vApps across infrastructure and the growing the vCloud Virtual Datacenter
  • vCloud Networking & Security VXLAN
  • Profile-Driven Storage integration with user and storage provided capabilities
  • Storage DRS (SDRS) integration
    • Exposes storage Pod as first class storage container (just like a datastores) making it visible in all workflows where a datastore is visible
    • Creation, modification, and deletion of spods not possible in vCD
    • Member datastore operations not permissible in VCD
  • Single level Snapshot & Revert support for vApps (create/revert/remove); integration with Chargeback
  • Integrated vShield Edge Gateway
  • Integrated vShield Edge Configuration
  • vCenter Single Sign-On (SSO)
  • New Features in Networking
    • Integrated Organization vDC Creation Workflow
    • Creates compute, storage, and networking objects in a single workflow
    • The Edge Gateway are exposed at Organization vDC level
    • Organization vDC networks replace Organization networks
    • Edge Gateways now support:
      • Multiple Interfaces on a Edge Gateway
      • The ability to sub-allocate IP pools to a Edge Gatewa
      • Load balancing
      • HA (not the same as vSphere HA)
        • Two edge VMs deployed in Active-Passive mode
        • Enabled at time of gateway creation
        • Can also be changed after the gateway has been completed
        • Gets deployed with first Organizational network created that uses this gateway
      • DNS Relay
        • Provides a user selectable checkbox to enable
        • If DNS servers are defined for the selected external network, DNS requests will be sent to the specified server. If not, then DNS requests will be sent to the default gateway of the external network.
      • Rate limiting on external interface
    • Organization networks replaced by Organization vDC Networks
      • Organization vDC Networks are associated with an Organization vDC
      • The network pool associated with Organization vDC is used to create routed and isolated Organization vDC networks
      • Can be shared across Organization vDCs in an Organization
    • Edge Gateways
      • Are associated with an Organization vDC, can not be shared across Organization vDCs
      • Can be connected to multiple external networks
        • Multiple routed Organization vDC networks will be connected to the same Edge Gateway
      • External network connectivity for the Organization vDC Network can be changed after creation by changing the external networks which the edge gateway is connected.
      • Allows IP pool of external networks to be sub-allocated to the Edge Gateway
        • Needs to be specified in case of NAT and Load Balancer
    • New Features in Gateway Services
      • Load balancer service on Edge Gateways
      • Ability to add multiple subnets to VPN tunnels
      • Ability to add multiple DHCP IP pools
      • Ability to add explicit SNAT and DNAT rules providing user with full control over address translation
      • IP range support in Firewall and NAT services
      • Service Configuration Changes
        • Services are configured on Edge Gateway instead of at the network level
        • DHCP can be configured on Isolated Organization vDC networks.
  • Usability Features
    • New default branding style
      • Cannot revert back to the Charcoal color scheme
      • Custom CSS files will require modification
    • Improved “Add vApp from Catalog” wizard workflow
    • Easy access to VM Quota and Lease Expirations
    • New dropdown menu that includes details and search
    • Redesigned catalog navigation and sub-entity hierarchy
    • Enhanced help and documentation links
  • Virtual Hardware Version 9
    • Supports features presented by HW9 (like 64 CPU support)
    • Supports Hardware Virtualization Calls
    • VT-x/EPT or AMD-V/RVI
    • Memory overhead increased, vMotion limited to like hardware
    • Enable/Disable exposed to users who have rights to create a vApp Template
  • Additional Guest OS Support
    • Windows 8
    • Mac OS 10.5, 10.6 and 10.7
  • Storage Independent of VM Feature
    • Added support for Independent Disks
    • Provides REST API support for actions on Independent Disks
      • As these consume disk space, the vCD UI was updated to show user when they are used:
      • Organizations List Page
      • A new Independent Disks count column is added.
      • Organization Properties Page
      • Independent Disks tab is added to show all independent disks belonging to vDC
      • Tab is not shown if no independent disk exists in the vDC.
      • Virtual Machine Properties Page
      • Hardware tab->Hard Disks section, attached independent disks are shown by their names and all fields for the disk are disabled as they are not editable.

That’s all I have time for right now.  As I said, there is more to come later on topics such as vDS enhancements, VXLAN, SRM, vCD Load Balancing, and vSphere Replication.  Stay tuned!

Using PowerCLI To Answer Virtual Machine Message Questions

April 17th, 2012

Scripters believe in efficiency, automation, working smarter and not harder, etc.  You’ll sometimes hear them say something like “Any process which is repeated once or more should be scripted”.  Myself, I am not the world’s greatest scripter.  I have to work at it.  Thus my personal threshold for committing to a scripted method is higher than one repetition.

In my environment, I did come across a particular process which needed to be repeated 20 or more times – Answering a virtual machine question.  Based on what I’m working on, conceivably I was going to run into this a lot more.  This is a great candidate for scripting.

Snagit Capture

The Virtual Machine Message I’m faced with deals with the relocation of the VM.  Via the vSphere Client, my possible choices to answer the question are “Cancel”, “I moved it”, or “I copied it”.  I don’t have the patience or desire to mouse through this hundreds of times.

Snagit Capture

I want to provide the same answer, “I moved it”, for every VM in inventory which has this question.  The script to accomplish this is fairly simple, even by my standards.  Once the PowerCLI connection is established to the vCenter Server or ESX(i) host, it’s a one-liner.  Following is the PowerShell script which gets the job done for my situation:

Connect-VIServer vc501.boche.lab
Get-VM | Get-VMQuestion | Set-VMQuestion -Option “I moved it” -Confirm:$false

Note that there are different types of Virtual Machine Message questions which will yield a different set of possible answers.  Be sure to query a VM having a question via PowerCLI for the possible answers to that question.  Get-VM | Get-VMQuestion -full should do it.  Once the possible answers are revealed, use Set-VMQuestion -Option to provide an answer.

Also note the script above will cycle through all VMs in inventory, and for those having a question, it will provide the same response for each VM.  Thus the assumption is made that all VMs with pending questions have the same question being asked.  To respond to explicit question types or to filter the VMs being looped through, the script would need to be refined.

For more information on the Get-VMQuestion or Set-VMQuestion PowerCLI cmdlets, use Get-Help Get-VMQuestion -full or Set-Help Get-VMQuestion -full respectively.

Update 6/30/15:  I ran into a situation with vSphere 5.5 Update 2 and PowerCLI 5.8.0.6734 where “I moved it” was not being recognized as a valid answer to the question, yet I needed to provide this as the correct answer.

Using:

PowerCLI C:\> Get-VMQuestion -VM mytestvm

Possible answers are:

Cancel, button.uuid.movedTheVM, button.uuid.copiedTheVM

I’m not sure why this changed or when, but in my case the new answer string to provide is ‘button.uuid.movedTheVM’

I needed to answer the same question on a large number of virtual machines so I used the following PowerShell:

Get-VM | Get-VMQuestion | Set-VMQuestion -Option ‘button.uuid.movedTheVM’ -Confirm:$false