Posts Tagged ‘Security’

vCloud Director 5.6.4 Remote consoleproxy issues

June 12th, 2015

vCloud Director is a wonderful IaaS addition to any lab, development, or production environment. When it’s working properly, it is a very satisfying experience wielding the power of agility, consistency, and efficiency vCD provides. However, like many things tech with upstream and human dependencies, it can and does break. Particularly in lab or lesser maintained environments that don’t get all the care and feeding production environments benefit from. When it breaks, it’s not nearly as much fun.

This week I ran into what seemed like a convergence of issues with vCD 5.6.4 relating to the Remote Console functionality in conjunction with SSL certificates, various browser types, networking, and 32-bit Java. As is the case often, what I’m documenting here is really more for my future benefit as there were a number of sparse areas I covered which I won’t necessarily retain in memory long but as it goes with blogs and information sharing, sharing is caring.

The starting point was a functional vCD 5.6.4-2496071 environment on vSphere 5.5. Everything historically and to date working normally with the exception of the vCD console which had stopped working recently in Firefox and Google Chrome browsers. Opening the console in either browser from seemingly any client workstation yielded the pop out console window with toolbar buttons along the top, but there was no guest OS console painted in the main window area. It was blank. The status of the console would almost immediately change to Disconnected. I’ve dealt with permutations of this in the past and I verified all of the usual suspects: NTP, DNS, LDAP, storage capacity, 32-bit Java version, blocked browser plug-ins, etc. No dice here.

In Firefox, the vCD console status shows Disconnected while the Inspect Element console shows repeated failed attempts to connect to the consoleproxy address.

10:11:30.195 "10:11:30 AM [TRACE] mks-connection: Connecting to wss://172.16.21.151/902;cst-t3A6SwOSPRuUqIz18QAM1Wrz6jDGlWrrTlaxH8k6aYuBKilv/1mc7ap50x3sPiHiSJYoVhyjlaVuf6vKfvDPAlq2yukO7qzHdfUTsWvgiZISK56Q4r/4ZkD7xWBltn15s5AvTSSHKsVbByMshNd9ABjBBzJMcqrVa8M02psr2muBmfro4ZySvRqn/kKRgBZhhQEjg6uAHaqwvz7VSX3MhnR6MCWbfO4KhxhImpQVFYVkGJ7panbjxSlXrAjEUif7roGPRfhESBGLpiiGe8cjfjb7TzqtMGCcKPO7NBxhgqU=-R5RVy5hiyYhV3Y4j4GZWSL+AiRyf/GoW7TkaQg==--tp-B5:85:69:FF:C3:0A:39:36:77:F0:4F:7C:CA:5F:FE:B1:67:21:61:53--"1 debug.js:18:12

10:11:30.263 Firefox can't establish a connection to the server at wss://172.16.21.151/902;cst-t3A6SwOSPRuUqIz18QAM1Wrz6jDGlWrrTlaxH8k6aYuBKilv/1mc7ap50x3sPiHiSJYoVhyjlaVuf6vKfvDPAlq2yukO7qzHdfUTsWvgiZISK56Q4r/4ZkD7xWBltn15s5AvTSSHKsVbByMshNd9ABjBBzJMcqrVa8M02psr2muBmfro4ZySvRqn/kKRgBZhhQEjg6uAHaqwvz7VSX3MhnR6MCWbfO4KhxhImpQVFYVkGJ7panbjxSlXrAjEUif7roGPRfhESBGLpiiGe8cjfjb7TzqtMGCcKPO7NBxhgqU=-R5RVy5hiyYhV3Y4j4GZWSL+AiRyf/GoW7TkaQg==--tp-B5:85:69:FF:C3:0A:39:36:77:F0:4F:7C:CA:5F:FE:B1:67:21:61:53--.1 wmks.js:321:0

tail -f /opt/vmware/vcloud-director/logs/vcloud-container-debug.log |grep consoleproxy revealed:
2015-06-12 10:50:54,808 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0x22c9c990 [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61719]] |
2015-06-12 10:50:54,854 | DEBUG    | consoleproxy              | ReadOperation                  | IOException while reading data: java.io.IOException: Broken pipe |
2015-06-12 10:50:54,855 | DEBUG    | consoleproxy              | ChannelContext                 | Closing channel java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61719] |
2015-06-12 10:50:55,595 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0xd191a58 [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61720]] |
2015-06-12 10:50:55,648 | DEBUG    | pool-consoleproxy-4-thread-289 | SSLHandshakeTask               | Exception during handshake: java.io.IOException: Broken pipe |
2015-06-12 10:50:56,949 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0x3f0c025b [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61721]] |
2015-06-12 10:50:57,003 | DEBUG    | pool-consoleproxy-4-thread-301 | SSLHandshakeTask               | Exception during handshake: java.io.IOException: Broken pipe |
2015-06-12 10:50:59,902 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0x1bda3590 [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61723]] |
2015-06-12 10:50:59,959 | DEBUG    | pool-consoleproxy-4-thread-295 | SSLHandshakeTask               | Exception during handshake: java.io.IOException: Broken pipe |

In Google Chrome, the vCD console status shows Disconnected while the Inspect element console (F12) shows repeated failed attempts to connect to the consoleproxy address.

10:26:43 AM [TRACE] init: attempting ticket acquisition for vm vcdclient
10:26:44 AM [TRACE] plugin: Connecting vm
10:26:44 AM [TRACE] mks-connection: Connecting to wss://172.16.21.151/902;cst-f2eeAr8lNU6BTmeVelt9L8VKoe92kJJMxZCC2watauBV6/x…fmI8Xg==--tp-B5:85:69:FF:C3:0A:39:36:77:F0:4F:7C:CA:5F:FE:B1:67:21:61:53--
WebSocket connection to 'wss://172.16.21.151/902;cst-f2eeAr8lNU6BTmeVelt9L8VKoe92kJJMxZCC2watauBV6/x…fmI8Xg==--tp-B5:85:69:FF:C3:0A:39:36:77:F0:4F:7C:CA:5F:FE:B1:67:21:61:53--' failed: WebSocket opening handshake was canceled
10:26:46 AM [ERROR] mks-console: Error occurred: [object Event]
10:26:46 AM [TRACE] mks-connection: Disconnected [object Object]

tail -f /opt/vmware/vcloud-director/logs/vcloud-container-debug.log |grep consoleproxy revealed:
2015-06-12 10:48:35,760 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0x55efffb3 [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61675]] |
2015-06-12 10:48:39,754 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0x3f123a13 [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61677]] |
2015-06-12 10:48:42,658 | DEBUG    | consoleproxy              | SimpleProxyConnectionHandler   | Initiated handling for channel 0x7793f0a [java.nio.channels.SocketChannel[connected local=/172.16.21.151:443 remote=/172.31.101.6:61679]] |

If you have acute attention to detail, you’ll notice the time stamps from the cell logs don’t correlate closely with the time stamps from the browser Inspect element console. Normally this would indicate time skew or an NTP issue which does cause major headaches with functionality but that’s by design here for my various screen captures and log examples aren’t from the exact same point in time. So it’s safe to move on.

Looking at the most recent vCloud Director For Service Providers installation documentation, I noticed a few things.

  1. Although I did upgrade vCD a few months ago to the most current build at the time, there’s a newer build available: 5.6.4-2619597
  2. Through repetition, I’ve gotten quite comfortable with the use of Java keytool and its parameters. However, additional parameters have been added to the recommended use of the tool. Noted going forward.
  3. VMware self signed certificates expire within three (3) months. Self signed certificates were in use in this environment. I haven’t noticed this behavior in the past nor has it presented itself as an issue but after a quick review, the self signed certificates generated a few months ago with the vCD upgrade had indeed expired recently.

At this point I was quite sure the expired certificates was the problem although it seemed strange the vCD portal was still usable while only the consoleproxy was giving me fits.  So I went through the two minute process of regenerating and installing new self signed certificates for both http and the consoleproxy.  The vCD installation guide more or less outlines this process as it is the same for a new cell installation as it is for replacing certificates. VMware also has a few KB articles which address it as well (1026309, 2014237). For those going through this process, you should really note the keytool parameter changes/additions in the vCD installation guide.

While I was at it, I also built a new replacement cell on a newer version of RHEL 6.5, performed the database upgrades, extended the self signed certificate default expiration from three months to three years, and I retired the older RHEL 6.4 cell. Fresh new cell. New certs. Ready to rock and roll.

Not so much. I still had the same problem with the console showing Disconnected. However, the Inspection element console for each browser are now indicating some new error message which I don’t have handy at the moment but basically it can’t talk to the consoleproxy adddress at all. I tried to ping the address and it was dead from a remote station point of view although it was quite alive at a RHEL 6.5 command prompt. Peters Virtual Notes had this one covered thankfully. According to https://access.redhat.com/site/solutions/53031, a small change is needed for the file /etc/sysctl.conf.

net.ipv4.conf.default.rp_filter = 1

must be changed to

net.ipv4.conf.default.rp_filter = 2

Success. Surely consoleproxy will work now. Unfortunately it still does not want to work. Back to the java.io.IOException: Broken pipe SSL handshake issues although the new certificate for vCD’s http address is registered and working fine (remembering again each vCD cell has two IP addresses, one for http access and one for consoleproxy functionality – each requires a trusted SSL certificate or an exception).

The last piece of the puzzle was something I have never had to do in the past and that is to manually add an exception (Firefox) for the consoleproxy self signed certificate and install it (Google Chrome). For each browser, this is a slightly different process.

For Firefox, browse to the https:// address of the consoleproxy, don’t worry, nothing visible should be displayed when it does not receive a properly formatted request. The key here is to add an exception for the certificate associated specifically to the consoleproxy address.

Once this certificate exception is added, the consoleproxy certificate is essentially trusted and so is the IP address for the host and the console service it is supposed to provide.

To resolve the consoleproxy issue for Google Chrome, the process is slightly different. Ironically I found it easiest to use Internet Explorer for this. Open Internet Explorer and when you do so, be sure to right click on the IE shortcut and Run as administrator (this is key in a moment). Browse to the https:// address of the consoleproxy, don’t worry, nothing visible should be displayed when it does not receive a properly formatted request. Continue to this website and then use the Certificate Error status message in the address bar to view the certificate being presented. The self signed consoleproxy certificate needs to be installed. Start this task using the Install Certificate button. This button is typically missing when launching IE normally but it is revealed when launching IE with Run as administrator rights.

Browse for the location to install the self signed certificate. Tick the box Show physical stores. Drill down under Third-Party Root Certification Authorities. Install the certificate in the Local Computer folder. This folder is typically missing when launching IE normally but it is revealed when launching IE with Run as administrator rights.

Once this certificate is installed, the consoleproxy certificate is essentially trusted in Google Chrome. Just as with the Firefox remedy, the Java SSL handshake with the consoleproxy succeeds and the vCD remote console is rendered.

Note that for Google Chrome, there is another quick method to temporarily gain functional console access without installing the consoleproxy certificate via Internet Explorer.

  1. Open a Google Chrome browser and browse to the https:// address of the consoleproxy.
  2. When prompted with Your connection is not private, click the Advanced link.
  3. Click the Proceed to (unsafe) link.
  4. Nothing will visibly happen except Google Chrome will now temporarily trust the consoleproxy certificate and the vCD remote console will function for as long as a Google Chrome tab remains open.
  5. Without closing Google Chrome, now continue into the vCD organization portal and resume business as usual with functional remote consoles.

On the topic of Google Chrome, internet searches will quickly reveal vCloud Director console issues with Google Chrome and NPAPI. VMware discusses this in the vCloud Director 5.5.2.1 Release Notes:

Attempts to open a virtual machine console on Google Chrome fail
When you attempt to open a virtual machine console on a Google Chrome browser, the operation fails. The occurs due to the deprication of NPAPI in Google Chrome. vCloud Director 5.5.2.1 uses WebMKS instead of the VMware Remote Console to open virtual machine consoles in Google Chrome, which resolves this issue.

I was working with vCD 5.6.x which leverages WebKMS in lieu of NPAPI so the NPAPI issue was not relevant in this case but if you are running into an NPAPI issue, Google offers How to temporarily enable NPAPI plugins here.

Update 8/8/15: Josiah points out a useful VMware forum thread which may help resolve this issue further when FQDNs are defined in DNS for remote console proxies or where multiple vCloud cells are installed in a cluster behind a front end load balancer, NAT/reverse proxy, or firewall.

VMware vSphere Hardening Guides

June 7th, 2014

Quick security related resource pointer on a Saturday morning. Over the years I’ve been collecting the various vSphere hardening guide documents as they are released.  These guides can be used to lock down your own (or your customer’s) environment to prevent or isolate security related breaches and to satisfy internal or external IT audits. Thanks to Mike Foley, I noticed the vSphere Hardening Guide 5.5 Update 1 was released yesterday. You’ll find adds/moves/changes in the following categories:

  • General (VCM, etc.)
  • SSO
  • ESXi
  • Virtual Machines
  • vCenter Server and VCSA
  • VUM (Update Manager)
  • vSphere Web Client

If you haven’t yet, grab the guide, take a look at it, and upgrade to vSphere 5.5 Update 1, hopefully in that order.

In the past I recall these guides were spread out on somewhat sparsely on VMware’s site. What I hadn’t noticed until this morning is that VMware has now compiled all available vSphere hardening guide links onto a single landing page in addition to providing change tracking between each of the vSphere 5.x guides which I think is quite helpful.

Failed to connect to VMware Lookup Service

March 14th, 2014

Judging by the search results returned by Google, it looks like my blog is among the few virtualization blogs remaining which does not have a writeup on this topic.  It’s Friday so… why not.

Scenario:  vSphere 5.5 Update 1 VMware vSphere Web Client fails to log into vCenter Server (appliance version) with the following error returned:

Failed to connect to VMware Lookup Service

https://fqdn:7444/lookupservice/sdk –

SSL certificate verification failed.

Snagit Capture

Contributing factors in my case which may have played a role in this once working environment:

  1. Recently upgraded vCenter 5.5.0 Server appliance to Update 1 (unlikely as other similar environments were not impacted after upgrade)
  2. This particular vCenter appliance was deployed as a vApp from a vCloud Director catalog (likely  but unknown at this time if a customization was possible or attempted during deployment)
  3. The hostname of the appliance may have been changed recently (very likely)

The solution is quite simple.

  1. Log into the vCenter Server appliance management interface (https://fqdn:5480/)
  2. Navigate to the Admin tab
  3. Certificate regeneration enabled: choose Yes
  4. Click the Submit button
  5. Navigate to the System tab
  6. Reboot the appliance

After the appliance reboots

  1. Log into the vCenter Server appliance management interface (https://fqdn:5480/)
  2. Navigate to the Admin tab
  3. Certificate regeneration enabled: choose No
  4. Click the Submit button
  5. Log out of the vCenter Server appliance management interface
  6. Log into the VMware vSphere Web Client normally

Admittedly I recalled the Certificate regeneration feature first by logging into the vCenter Server appliance management interface, but then verified with a search to ensure the purpose of the Certificate regeneration feature.  The search results turned up Failed to connect to VMware Lookup Service – SSL Certificate Verification Failed (among many other blog posts as mentioned earlier) in addition to VMware KB 20333338 Troubleshooting the vCenter Server Appliance with Single Sign-On login.  Both more or less highlight a discrepancy between the appliance hostname and the SSL certificate resulting in the need to regenerate the certificate to match the currently assigned hostname.

I ran across another issue this week during the Update 1 upgrade to the vCenter appliance which I may or may not get to writing about today.

At any rate, have wonderful and Software Defined weekend!

vCenter Server Appliance 5.5 root account locked out after password expiration

January 10th, 2014

Thanks to Chris Colotti, I learned of a new VMware KB article today which could potentially have wide spread impact, particularly in lab, development, or proof of concept environments.  The VMware KB article number is 2069041 and it is titled The vCenter Server Appliance 5.5 root account locked out after password expiration.

In summary, the root account of the vCenter Server Appliance version 5.5 becomes locked out 90 days after deployment or root account password change.  This behavior is by design which follows a security best practice of password rotation.  In this case, the required password rotation interval is 90 days after which the account will be forcefully locked out if not changed.

The KB article describes processes to prevent a forced lockout as well as unlocking a locked out root account.

Approximately 90 days have elapsed since the release of vSphere 5.5 and I imagine this issue will quickly begin surfacing in large numbers where the vCenter Server Appliance 5.5 has been deployed using system defaults.

Update 6/16/16: For more information on vCenter Server Appliance password policies, including the local root account, check out vCSA 6.0 tricks: shell access, password expiration and certificate warnings.

Single Sign-On Warning 25000

November 12th, 2013

Up to this point, I’ve deployed several net new instances of vCenter Server 5.5 and of course its essential components including Single Sign-On, Inventory Service, next generation Web Client, and the legacy vSphere Client.  Most of these deployments leveraged the vCenter appliance.  Using the appliance is a very easy to deploy vCenter because all of the essential components are pre-installed in the appliance and only need to be configured.

One area I hadn’t tackled much yet are upgrades of existing Windows-based vCenter environments to vSphere 5.5.  Having recently completed an inline upgrade of vCloud Director 5.1.2 to 5.5, it was now time to upgrade said vCloud’s underlying vSphere 5.1 (Update 1a I believe) virtual infrastructure.   Prior to starting the upgrade, I took the necessary precautions of getting a point in time snapshot of the vCenter Server, the vCloud Director Cells, and the Microsoft SQL Server databases for each (three total: SSO, vCenter, and vCD).  I accomplished this using array based snapshots – in this case Dell Compellent Storage Center Replays.

I launched autorun from the vCenter 5.5 installation media.  I opted for the custom installation and started with the Single Sign-On (SSO) upgrade from 5.1 to 5.5.  During the installation, I was met with

Warning 25000.  Please verify that the SSL certificate for your vCenter Single Sign-On 5.1 SSL is not expired.  If it did expire, please replace it with a valid certificate before upgrading to vCenter Single Sign-On 5.5.

Snagit Capture

In this particular environment, self-signed certificates from VMware were in use.  I know that this environment was deployed new less than two years ago and a verification of the SSL certificates in use proved that none were expired.  But because SSO and vCenter are such integral components to vCloud Director, I didn’t want to proceed without further vetting this out.

Google.

Upgrade from vSphere 5.1 to vSphere 5.5 rolls back after importing Lookup Service data (2060511) – This KB article describes a situation in which Warning 25000 results when a registry value on the existing Windows-based SSO 5.1 server does not match a field on the SSL certificate.  The resolution involves simply changing the registry value to match that which is on the SSL certificate.  I won’t repeat the details because you can read the KB article yourself.  Furthermore it didn’t resolve the problem in this case because the field on my SSL certificate and the registry key were an identical match.

Upgrading to VMware vCenter Single Sign-On 5.5 displays the error: Warning 25000 (2061478) – This KB article describes a problem for which there is no resolution. However, there is a workaround and that involves changing service_id and service.properties files.  More detail is available in the KB article and again the symptoms in the log files weren’t a close match.

The Trouble With SSL Certificates and Upgrading to VMware SSO 5.5 – Then I took a look at Michael Webster’s blog article on precisely the same error message.  Michael briefly discusses the two SSL certificate deployment models and then digs into VMware KB 2060511 mentioned above.  While the information in Michael’s blog article reassured me I was not alone in my journey, KB 2060511 didn’t solve my problem either.  But sometimes the value of blog articles is not only in the original author’s content, but also in the follow up comments from the readers.  Such was the case here.  A number of Michael’s readers responded by saying they were essentially in the same boat I’m in – it sounds like KB 2060511, but in the end this article doesn’t have the solution because there was nothing wrong with their SSO registry values.  The readers found no choice but to push onward beyond Warning 25000 with fingers crossed.  As it turned out in my as well as with some others, Warning 25000 was benign in nature and the installation completed successfully with no rollback.

In summary, this blog post does not represent global authority to ignore Warning 25000.  Rather it is meant to highlight one particular scenario where Warning 25000 may present itself and the actions that were taken to work through the problem.  I can’t stress enough the importance of the SSO component of vCenter going forward.  If any conclusion can be drawn here, it is that a backup of the infrastructure components should be secured before committing to the upgrade steps.  In this case, snapshots are the quickest and easiest method to provide data protection and recovery.  Although vSphere snapshots would work in some deployment architectures, recovering an environment when the environment being upgraded is managing the snapshots could be a challenge.  That is why I chose an out of band array based snapshot in this instance.

I would also like to point out in closing that vSphere 5.5 is still relatively new and VMware appears to still be chasing down all possible causes, resolutions, and workarounds to Warning 25000.  New information as well as VMware KB articles may develop subsequent to this writing so it may be worth continuing your own Google searching beyond this point.

Have a great week!

A Look At vCenter 5.5 SSO RC Installation

August 30th, 2013

This week at VMworld 2013, I attended a few sessions directly related to vCenter 5.5 as well as its components, one of which is vCenter Single Sign On (SSO):

  • VSVC5234 – Extreme Performance Series: vCenter of the Universe
  • VSVC4830 – vCenter Deep Dive

First of all, both sessions were excellent and I highly recommend viewing them if you have access to the post conference recordings. 

If you followed my session tweets or if perhaps you’ve read half a dozen or more already available blog posts on the subject, you know that several improvements have been made to vCenter SSO for the vSphere 5.5 release.  For instance:

  • Completely re-written from the ground
  • Multi-master architecture
  • Native replication mechanism
  • SSO now has site awareness (think of the possibilities for HA stretched clusters)
  • MMC based diagnostic suite available as a separately maintained download
  • The external database and its preparation dependency has been removed
  • Database patitioning to improve both scalability and performance (this was actually added in 5.1 but I wanted to call it out)
  • Revamped multi-site deployment architecture
  • Full Mac OS X web client support including remote console
  • Improved certificate management
  • Multi-tenant capabilities
  • Drag ‘n’ Drop in the 5.5 web client

With some of the new features now identified and VMware’s blessing, have a look at the installation screens and see if you can spot the differences as compared to a vCenter 5.1 SSO installation.  These stem from a manual installation of SSO, not an automated installation of all vCenter components (by the way, the next gen web client is now installed as part of an automated vCenter 5.5 installation whereas it was not in 5.1).  Keep in mind these were pulled from a release candidate version and may change when vCenter 5.5 GAs at a future date.

I noticed one subtle change here – clicking on the Microsoft .NET 3.5 SP1 link in Windows 2008R2 actually installs the feature rather than just throwing up a dialogue box asking you to install the feature yourself.

Snagit Capture

As this is a manual installation, we have the option to use the default or specify the installation location.  Best practice is to install all vCenter components together so that they can communicate at server bus speed and won’t be impacted by network latency.  However, for larger scale environments, SSO should be isolated on a separate server with five or more vCenter Servers in the environment.  On a somewhat related note, the Inventory Service may benefit from an installation on SSD, again in large infrastructures.

Snagit Capture

We won’t likely see this in the GA version.

Snagit Capture

We’re going through the process of installing vCenter version 5.5 but in terms of the SSO component, again this is a complete re-write and bears the respective version of 2.0.

Snagit Capture

We always read the EULA in full and agree to the license terms and conditions.

Snagit Capture

Snagit Capture

Big changes here.  Note the differences in the deployment models compared to the previous 5.1 version – previous deployment models are honored through an upgrade to 5.5.  Again, this is where the VMworld sessions noted above really go into detail. 

Snagit Capture

the System-Domain namespace has been replaced with vsphere.local.

Snagit Capture

The new site awareness begins here.

Snagit Capture

Snagit Capture

Snagit Capture

Snagit Capture

I hope you agree that SSO installation in vCenter 5.5 has been simplified while many new features have been added at the same time.

As always, thank you for reading and it was a pleasure to meet and see everyone again this year at VMworld.

View 5.1 Upgrade Experience. Composer, Permissions, and SSL – Oh My!

August 8th, 2012

The other night I upgraded the VMware View 5.0.1 environment in the lab to 5.1 which was released on May 16th.  Normally when I upgrade the View environment, I don’t actually perform an inline upgrade of the Connection Server or database.  The environment is small enough that I can flatten it and rebuild fresh from scratch (including brand new VMs for the infrastructure components such as the Connection Server) for each new version VMware releases.  Due to VMware’s aggressive release schedule, I also embed the production version in the infrastructure server name which helps me keep track of where things are at in the lab.  Thus, with each new release, I’m building new infrastructure VMs with updated names, rather than recycling the previous infrastructure VMs, renaming them, remove/re-add to the domain, and even then I’m left with a VM name which doesn’t match the name on the datastore folder.  Pushing the reset button and starting fresh obliterates any bad DNA or cooties the previous environment might have had and it gives me a little extra peace of mind when I sleep at night.

I was running a little short on time so for this round I decided to perform an inline upgrade to 5.1 rather than going through the normal rebuild routine.  After all, most production environments don’t have the luxury of starting over so now was as good a time as ever to test the upgrade process of View in the lab.  Again – a fairly simple setup: a Connection Server, View Composer 2.7 installed on the vCenter Server which for the first time in many releases will be upgraded to 3.0, back end databases on an external SQL server, and 3 small pools.

The View Connection Server upgrade went as planned. No issues to speak of there (yet).  However, I did struggle with the View Composer upgrade.  The first run through uninstalled View Composer and failed with an error which I wasn’t quick enough to capture.  I re-ran through the Composer installation and it failed again with the same error:

The wizard was interrupted before VMware View Composer could be completely installed.

While I was perfrming some troubleshooting, a couple of gracious folks on Twitter by the name of Diego Quintana and Tim Washburn (@daquintana and @mittim12 respectively) pointed out VMware KB article 2017773 Installing or upgrading View Composer fails with error: The wizard was interrupted before VMware View Composer could be completely installed which resolved my issue.  The previous View Composer installation had placed one or more keys in the directory C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\ which my user account no longer had NTFS permissions to.  The resolution was to simply relax the NTFS permissions both on the MachineKeys folder as well as the files inside of the folder for good measure.

I thought I was out of the woods, but not quite yet.  SSL certificate issues followed.

Snagit Capture

VMware made some new changes with regards to SSL in View 5.1 which are documented at :\Program Files\VMware\VMware View\Server\README.rtf

Copied and pasted verbatim, the release notes are:

Read These Notes!  Your View 5.1 Setup Will Be Easier!

You can read these notes in your language:
Français    Deutsch     简体中文     日本語     한국어

We made changes in View 5.1 that require you to configure View components a little differently than in the past.  These notes will help you to avoid potential pitfalls when you install or upgrade to View 5.1.

1)  You cannot downgrade View 5.1 Connection Server to previous versions.

In View 5.1, the View LDAP configuration is encrypted and cannot be used by earlier versions of View.

  • After you upgrade a View Connection Server instance to View 5.1, you cannot downgrade that instance to an earlier version.
  • After you upgrade all View Connection Server instances in a replicated group, you cannot add another instance that runs an earlier version of View.

Note: Downgrading was never supported, but in past releases it worked.  Now it won’t work.

2)  vCenter Server and View Composer hosts need valid SSL certificates.

  • Best choice: Ensure your vCenter Server and View Composer have Certificate Authority (CA)-provided certificates:

o  Install an SSL certificate, signed by a CA, on the Windows Server on which vCenter Server is installed.

o  Do the same for View Composer. If you install View Composer and vCenter Server on the same host, they can use the same certificate, but you must configure the certificate separately for each component.

* If you install the certificate before you install View Composer, you can select your certificate during the View Composer installation.

* If you replace the default certificate later, run the SviConfig ReplaceCertificate command to bind the new certificate to the port used by View Composer.

o  Make sure the CA for the new certificates, and any parent CAs, are trusted by each Windows server on which a View Connection Server instance is installed.

  • Alternative: After you add vCenter Server and View Composer to View, accept the thumbprint of the default certificate for View Composer by clicking Verify in View Administrator.  Do the same for vCenter Server.

More information: See “Configuring SSL Certificates for View Servers” in the View Installation guide.

3)  Security server and View Connection Server hosts need valid SSL certificates.

  • Best choice: After you install a View Connection Server instance or security server on a Windows Server host, open the Windows Server certificate store and take these steps:

o  Import an SSL certificate that is signed by a CA and that your clients can validate.

o  Make sure that the entire certificate chain, including intermediate certificates and root certificate, are installed.

o  Make sure the certificate has a private key, and mark the key as exportable.

o  Configure the certificate Friendly Name as vdm.

  • Alternative: Let the View server installer create a default certificate in the Windows Server certificate store. The certificate is self-signed and will be shown as invalid in View Administrator.
  • Upgrading to View 5.1: If your original View servers already have SSL certificates signed by a CA, you don’t have to do anything.  During the upgrade, View imports your certificates into the Windows Server certificate store.

If your original View servers have default certificates, upgrade your View servers and follow the Best choice steps shown above.

More information: See “Configuring SSL Certificates for View Servers” in the View Installation guide.

4)  Certificates for vCenter Server, View Composer, and View servers must include certificate revocation lists (CRLs).

View will not validate a certificate without a CRL.

  • Best Choice: lf needed, take these steps:

o  Add a CRL to your certificate.

o  Import the updated certificate into the Windows certificate store on the vCenter Server, View Composer, and View server host.

  • Alternative: Change the registry settings that control CRL checking.

More information: “Configuring Certificate Revocation Checking on Server Certificates” in the View Installation guide.

5)  Windows Firewall with Advanced Security must be enabled on Security Server and View Connection Server hosts. 

By default, IPsec rules govern connections between the View security server and View Connection Server and require Windows Firewall with Advanced Security to be enabled.

  • Best choice: Set Windows Firewall with Advanced Security to on before you install the View servers. Make sure it’s on for any active profiles; better still, set it to on for all profiles.
  • Alternative: Before you install security servers, open View Administrator and disable the Global Setting, Use IPsec for Security Server Connections, by setting it to no. (This is not recommended.)

6)  Back-end firewalls must be set up to support IPsec.

If you have a back-end firewall between security servers and View Connection Server instances, you must configure firewall rules to allow the connections to work.

More information: See “Configuring a Back-End Firewall to Support IPsec ” in the View Installation guide.

7)  View Clients must use HTTPS to connect to View.

View Connection Server instances and security servers use SSL for client connections.

  • If View clients connect via an SSL off-loading intermediate device, you must install the intermediate device’s SSL certificate on View Connection Server or security server.
  • The connection must be HTTPS whether or not a View client connects via an intermediate device such as a load balancer. If you use an intermediate device, and you want the connection between the intermediate device and View server to be over HTTP (SSL off-loading), configure the locked.properties file on the View server.
  • Older View clients that can choose not to use HTTPS will get an error if users select HTTP. Previously they were silently redirected to HTTPS. Clients that cannot make SSL connections will be unable to connect to View.

More information: See “Off-loading SSL Connections to Intermediate Servers” in the View Administration guide.

8)  Encrypted and cleansed View backups require new restore steps.

By default, View 5.1 backups are encrypted. You can also cleanse View backups (exclude passwords and other sensitive information from the backup data) or back up in plain text (not recommended).

  • To restore an encrypted backup, you must decrypt the data first. You must use the data recovery password that you provided when you installed View Connection Server.
  • Do not restore cleansed backups. Data such as passwords will be missing from your View LDAP configuration. View components will not function properly without this data. To restore normal functionality, you will have to use View Administrator to manually reset all passwords and other missing data items.

More information: See “Backing Up and Restoring View Configuration Data” in the View Administration guide.

9)  Before you can upgrade or reinstall a View 5.1 security server, you must remove the relevant IPsec rules from the paired View Connection Server instance so that fresh rules can be established.

  • In View Administrator, select the security server and click More Commands > Prepare for Upgrade or Reinstallation.

Note: You don’t need to remove a security server from View before you upgrade or reinstall the server.

More information: See “Prepare to Upgrade or Reinstall a Security Server” in the View Installation guide.

Ok, so basically VMware is pushing for the use of SSL certificates from a trusted CA whether that be externally (VeriSign, etc.) or internally (Microsoft Certificate Services) generated.  For the time being, I have ditched my internal Microsoft CA and wish to continue using the self signed certificates shipped and installed by View.  To do so, as explained in the README above, one must visit the System Health in the View Administrator Dashboard and verify the certificates for the vCenter Server as well as the View Composer Server (each will be seen in a red status in the dashboard).  The Connection Server certificate cannot be verified and will remain in a red status however from this point forward both the View Connection Server and View Composer will function normally.

Upgrading the View Agents and recomposing the pools was a non-issue and the upgrade was completed successfully.  After all is said and done, the environment is working and the upgrade was successful.  View 5.0.1 Clients have no problem connecting to the new 5.1 environment; I’ll get the clients upgraded in the near future and I’ll consider resurrecting the lab CA to generate trusted SSL certificates.