Posts Tagged ‘Microsoft’

MegaSceneryEarth automated HTTP download PowerShell script

July 30th, 2018

Regulars: Thank you for stopping by but just as a heads up, this post is not VMware virtualization related.

After a bit of hardware trouble from the local store, my replacement flight sim rig is up and running Lockheed Martin Prepar3D 4.3. I’m trying to shake the rust off after not having flown a leg for quite some time but everything sim and add-on related is looking good and I was able to fly KMSP-KLAX with many more FPS that I’ve been used to the past many years on FSX.

Now, out of sheer coincidence late last week I received a promotion from MegaSceneryEarth. Normally I ignore these but since this was a 50% off deal and I’m essentially starting from ground zero with no scenery for Prepar3D, I took a look. After some chin scratching, I checked out with Minnesota, Arizona, Northern California, and Southern California. I felt like quite the opportunist until the email arrived with file download instructions. Each scenery is broken up into 2GB download chunks. Between the four sceneries, I was presented with 123 download links. With no FTP option that I’m aware of, I’m going to have to babysit this process. With each 2GB download taking anywhere from 5-10 minutes, this is going to feel a lot like installing Microsoft Office 2016 via floppy disk. Except longer. MegaSceneryEarth is available on shipped DVDs or USB sticks they cost a small fortune and of course the shipping and handling time. This is going to be painful.

But… there’s a cheaper, faster, and fully automated way. PowerShell scripting.

Now admittedly for my first scenery, I threw the v1.0 PowerShell script version together quickly using the Invoke-WebRequest cmdlet with my own static scenery-specific lines of code to serve my own needs. And it worked. I launched the script, walked away for a few hours, and when I came back I had a directory full of MegaSceneryEarth .zip files ready for installation. It worked beautifully and it didn’t consume bunches of my time.

I still had three sceneries left to download. While I could have made new static scripts for the remaining three sceneries, my mind was already going down the road that this script would be more powerful if one script worked on a variety of MegaSceneryEarth downloads. It becomes flexible to use for my future MegaSceneryEarth sceneries as well as extensible if shared with the community.

So on Sunday, version 2.0 is what I came up with. It takes three inputs in the form of modifying variables at the beginning of the script before running it:

  1. The download URL for the first file of the scenery
  2. The path on the local hard drive where all of the .zip files should be downloaded to
  3. The starting point or first file to download. Usually this will be 1 but if for some reason a previous download was stopped or interrupted at say 6, we’d want to continue from there assuming we don’t want to re-download the first 5 files all over again.

The script does the rest of the heavy lifting. All three of the above are required for functionality but by far the first variable is the most important and where I spent almost all of my time making the script flexible so that it works across MegaSceneryEarth sceneries. When provided with the download URL of the first file of the scenery, the script does a bunch of parsing, splitting, and concatenation of stuff to automatically determine some things:

  1. It figures out what the base path and file name URL is for the scenery. For any given scenery, these parts never change across all of the download links within a scenery but they will change across sceneries.
  2. It figures out how many .zip files there are to download in total for the entire scenery.
  3. Because MegaSceneryEarth likes to use leading zeroes as part of their naming convention, and leading zero strings aren’t the same thing as leading zero integers in terms of PowerShell constructs, it figures out how to deal with those for file names and looping iterations. Basically when you go from “009” files in a scenery download to “010”, without dealing with that, stuff breaks. I think most MegaSceneryEarth sceneries have more than 9 files, but some have less (Deleware has 1). The script deals with both kinds. That I know of, MegaSceneryEarth doesn’t have a scenery with more than 99 .zip files but version 2.1 of the script will accommodate that.
  4. It figures out the correct file name to save on the local hard drive for each scenery file downloaded. This may sound stupid but that I know of, the Invoke-WebRequest cmdlet requires the output file name and doesn’t automatically assume the file name should be the same as the file name being downloaded via HTTP such as a copy command to a folder would.

Generally speaking, for automation to work properly, it relies on input consistency so that all assumptions it makes are 100% correct. Translated, this script relies on consistent file/path naming conventions used by MegaSceneryEarth. Not only within sceneries, but across sceneries as well.

And so here it is. Copy the PowerShell script below and execute in your favorite PowerShell environment. Most of the time I simply use Windows PowerShell ISE. The code isn’t signed so you may need to temporarily relax your PowerShell ExecutionPolicy to Bypass . It’s there to protect you by default.

######################################################################################################################
# MegaSceneryEarth.ps1 version 2.1 7/31/18
#
# This PowerShell script automates unattended HTTP download of up to 999 MegaSceneryEarth scenery files
# Relies on MegaSceneryEarth naming conventions current as of version date above
# Tested successfully with four MegaSceneryEarth downloads
# As with anything on the internet, use at your own risk, no warranty provided by the author
#
# Update the three variables below as necessary for your specific scenery and download parameters
#
# Jason Boche
# http://boche.net
# jason@boche.net
######################################################################################################################

# Provide the first file download link copied from the MegaSceneryEarth .htm link provided with license key email
# Internet Explorer: Right click the download link and choose "Copy shortcut"
# Google Chrome: Right click the download link and choose "Copy link address"
# Then paste the link between the quotation marks

$firstfile1 = "paste first file download link here.zip"

# Provide the path on your hard drive where the files will temporarily be saved.
# This will usually be where you have the MegaSceneryEarthInstallManager.exe file for subsequent installation
# Syntax requires a trailing backslash (\)

$dlpath = "D:\junk\"

# Provide the starting file count. Usually we start with 1 unless you're starting the download somewhere after 1

$startcount = 1

# Do not edit anything below this line

######################################################################################################################

# Get the HTTP path
$firstfile2 = split-path $firstfile1
$firstfile3 = $firstfile2.Replace("\","/")
$httppath = $firstfile3 + "/"

# Get the .zip file name
$firstfile4 = split-path $firstfile1 -leaf

# Get the static part of the .zip file name
$firstfile5 = $firstfile4 -split "001_of_"
$firstfile6 = $firstfile5[0]
$fileprefix = $firstfile6

# Get the total number of scenery files in string format with and without .zip extension
$firstfile7 = $firstfile5[1] -split ".zip"
$firstfile8 = $firstfile5[1]
# Convert total number of scenery files in string format to number format
$total = [decimal]::Parse($firstfile7)

if ($startcount -gt $total)
{
Write-Host Error: Starting file specified is greater than total number of files. Please check the `$startcount variable
pause
exit
}

# If there are 1-9 scenery files, download those
if ($total -ge 1)
{
$leadingzero = "00"
while ($startcount -le $total)
{
if ($startcount -gt 9){break}
Write-Host Downloading $httppath$fileprefix$leadingzero$startcount"_of_"$firstfile8 to $dlpath$fileprefix$leadingzero$startcount"_of_"$firstfile8

Invoke-WebRequest -Uri $httppath$fileprefix$leadingzero$startcount"_of_"$firstfile8 -OutFile $dlpath$fileprefix$leadingzero$startcount"_of_"$firstfile8

$startcount ++

}
}

# If there are 10-99 scenery files, download those also
if ($total -ge 10)
{
$leadingzero = "0"
while ($startcount -le $total)
{
if ($startcount -gt 99){break}
Write-Host Downloading $httppath$fileprefix$leadingzero$startcount"_of_"$firstfile8 to $dlpath$fileprefix$leadingzero$startcount"_of_"$firstfile8

Invoke-WebRequest -Uri $httppath$fileprefix$leadingzero$startcount"_of_"$firstfile8 -OutFile $dlpath$fileprefix$leadingzero$startcount"_of_"$firstfile8

$startcount ++

}
}

# If there are 100-999 scenery files, download those also
if ($total -ge 100)
{
$leadingzero = ""
while ($startcount -le $total)
{
if ($startcount -gt 999){break}
Write-Host Downloading $httppath$fileprefix$leadingzero$startcount"_of_"$firstfile8 to $dlpath$fileprefix$leadingzero$startcount"_of_"$firstfile8

Invoke-WebRequest -Uri $httppath$fileprefix$leadingzero$startcount"_of_"$firstfile8 -OutFile $dlpath$fileprefix$leadingzero$startcount"_of_"$firstfile8

$startcount ++

}
}

VMware Horizon Share Folders Issue with Windows 10

June 12th, 2017

I spent some time the last few weekends making various updates and changes to the lab. Too numerous and not all that paramount to go into detail here, with the exception of one issue I did run into. I created a new VMware Horizon pool consisting of Windows 10 Enterprise, Version 1703 (Creators Update). The VM has 4GB RAM and VMware Horizon Agent 7.1.0.5170901 is installed. This is all key information contributing to my new problem which is the Shared Folders feature seems to have stopped functioning.

That is to say, when launching my virtual desktop from the Horizon Client, there are no shared folders or drives being passed through from where I launched the Horizon Client. Furthermore, the Share Folders menu item is completely missing from the blue Horizon Client pulldown menu.

I threw something out on Twitter and received a quick response from a very helpful VMware Developer by the name of Adam Gross (@grossag).

Adam went on to explain that the issue stems from a registry value defining an amount of memory which is less that the amount of RAM configured in the VM.

The registry key is HKLM\SYSTEM\CurrentControlSet\Control\ and the value configured for SvcHostSplitThresholdInKB is 3670016 (380000 Hex). The 3670016 is expressed in KB which comes out to be 3.5GB. The default Windows 10 VM configuration is deployed with 4GB of RAM which is what I did this past weekend. Since 3.5GB is less than 4GB, the bug rears its head.

Adam mentioned the upcoming 7.2 agent will configure this value at 32GB on Windows 10 virtual machines (that’s 33554432 or 2000000 in Hex) and perhaps even larger in the 7.2 version or some future release of the agent because the reality some day is that 32GB won’t be large enough. Adam went on to explain the maximum amount of RAM supported by Windows 10 x64 is 2TB which comes out to be 2147483648 expressed in KB or 80000000 in Hex. Therefore, it is guaranteed safe (at least to avoid this issue) to set the registry value to 80000001 (in Hex) or higher for any vRAM configuration.

To move on, the value needs to be tweaked manually in the registry. I’ll set mine to 32GB as I’ll likely never have a VDI desktop deployed between now and when the 7.2 agent ships and is installed in my lab.

And the result for posterity.

I found a reboot of the Windows 10 VM was required before the registry change made the positive impact I was looking for. After all was said and done, my shared folders came back as did the menu item from the pulldown on the blue Horizon Client pulldown menu. Easy fix for a rather obscure issue. Once again my thanks to Adam Gross for providing the solution.

VMware Tools causes virtual machine snapshot with quiesce error

July 30th, 2016

Last week I was made aware of an issue a customer in the field was having with a data protection strategy using array-based snapshots which were in turn leveraging VMware vSphere snapshots with VSS quiesce of Windows VMs. The problem began after installing VMware Tools version 10.0.0 build-3000743 (reported as version 10240 in the vSphere Web Client) which I believe is the version shipped in ESXI 6.0 Update 1b (reported as version 6.0.0, build 3380124 in the vSphere Web Client).

The issue is that creating a VMware virtual machine snapshot with VSS integration fails. The virtual machine disk configuration is simply two .vmdks on a VMFS-5 datastore but I doubt the symptoms are limited only to that configuration.

The failure message shown in the vSphere Web Client is “Cannot quiesce this virtual machine because VMware Tools is not currently available.”  The vmware.log file for the virtual machine also shows the following:

2016-07-29T19:26:47.378Z| vmx| I120: SnapshotVMX_TakeSnapshot start: ‘jgb’, deviceState=0, lazy=0, logging=0, quiesced=1, forceNative=0, tryNative=1, saveAllocMaps=0 cb=1DE2F730, cbData=32603710
2016-07-29T19:26:47.407Z| vmx| I120: DISKLIB-LIB_CREATE : DiskLibCreateCreateParam: vmfsSparse grain size is set to 1 for ‘/vmfs/volumes/51af837d-784bc8bc-0f43-e0db550a0c26/rmvm02/rmvm02-000001.
2016-07-29T19:26:47.408Z| vmx| I120: DISKLIB-LIB_CREATE : DiskLibCreateCreateParam: vmfsSparse grain size is set to 1 for ‘/vmfs/volumes/51af837d-784bc8bc-0f43-e0db550a0c26/rmvm02/rmvm02_1-00000
2016-07-29T19:26:47.408Z| vmx| I120: SNAPSHOT: SnapshotPrepareTakeDoneCB: Prepare phase complete (The operation completed successfully).
2016-07-29T19:26:56.292Z| vmx| I120: GuestRpcSendTimedOut: message to toolbox timed out.
2016-07-29T19:27:07.790Z| vcpu-0| I120: Tools: Tools heartbeat timeout.
2016-07-29T19:27:11.294Z| vmx| I120: GuestRpcSendTimedOut: message to toolbox timed out.
2016-07-29T19:27:17.417Z| vmx| I120: GuestRpcSendTimedOut: message to toolbox timed out.
2016-07-29T19:27:17.417Z| vmx| I120: Msg_Post: Warning
2016-07-29T19:27:17.417Z| vmx| I120: [msg.snapshot.quiesce.rpc_timeout] A timeout occurred while communicating with VMware Tools in the virtual machine.
2016-07-29T19:27:17.417Z| vmx| I120: —————————————-
2016-07-29T19:27:17.420Z| vmx| I120: Vigor_MessageRevoke: message ‘msg.snapshot.quiesce.rpc_timeout’ (seq 10949920) is revoked
2016-07-29T19:27:17.420Z| vmx| I120: ToolsBackup: changing quiesce state: IDLE -> DONE
2016-07-29T19:27:17.420Z| vmx| I120: SnapshotVMXTakeSnapshotComplete: Done with snapshot ‘jgb’: 0
2016-07-29T19:27:17.420Z| vmx| I120: SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Failed to quiesce the virtual machine (31).
2016-07-29T19:27:17.420Z| vmx| I120: VigorTransport_ServerSendResponse opID=ffd663ae-5b7b-49f5-9f1c-f2135ced62c0-95-ngc-ea-d6-adfa seq=12848: Completed Snapshot request.
2016-07-29T19:27:26.297Z| vmx| I120: GuestRpcSendTimedOut: message to toolbox timed out.

After performing some digging, I found VMware had released VMware Tools version 10.0.9 on June 6, 2016. The release notes identify the root cause has been identified and resolved.

Resolved Issues

Attempts to take a quiesced snapshot in a Windows Guest OS fails
Attempts to take a quiesced snapshot after booting a Windows Guest OS fails

After downloading and upgrading VMware Tools version 10.0.9 build-3917699 (reported as version 10249 in the vSphere Web Client), the customer’s problem was resolved. Since the faulty version of VMware Tools was embedded in the customer’s templates used to deploy virtual machines throughout the datacenter, there were a number of VMs needing their VMware Tools upgraded, as well as the templates themselves.

Error 1603: Java Update did not complete

February 9th, 2016

3 Billion Devices Run Java.

Unfortunately (or fortunately depending on how you look at it) my workstation isn’t one of them.

I’m no stranger to Java and I’m more than willing to share my bitter experiences with it but rarely does my dissatisfaction stem from the installation process itself (that is if you don’t count the update frequency). This blog post is for my future self as I’m bound to run into it again.

I encountered a never before seen by me problem installing the Windows Offline (64-bit) version of Java Version 8 Update 73 on Windows 7.

Java Update did not complete

Error Code: 1603

The usual tricks did not work. Uninstalling the old version first. Run as administrator. Directory and registry scrubbing. System reboot. Trying an older version. In fact all version 8 builds exhibited this behavior. It wasn’t until I rolled all the way back to a version 7 build that the installation was successful.

For posterity, this spiceworks thread covers a lot of ground; there seems to be a variety of fixes for an equal number of root causes which all yield the same Java installation failure. The Java knowledgebase covering this issue offers some basic workarounds under Option 1 such as a system reboot, offline installer, uninstalling old versions, all of which I had already tried without success.

The fix in my particular instance was Option 2: Disable Java content (in the web browser) through the Java Control Panel. You’ll find the Java applet in the Windows Control Panel when a 32-bit version of Java is installed.

On the Security tab, temporarily deselect the “Enable Java content in the browser” checkbox.

Complete the 64-bit version of Java installation (launch the installer using “Run as Administrator“) and don’t forget to “Enable Java content in the browser” when finished.

Update 5-12-16: If you’re receiving this error and “Enable Java content in the browser” is already deselected, check the box, Apply, OK, uncheck the box, Apply, OK.

 

vCloud Director Error Cannot delete network pool

August 15th, 2015

I ran into a small problem this week in vCloud Director whereby I was unable to Delete a Network Pool. The error message stated Cannot delete network pool because It is still in use. It went on to list In use items along with a moref identifier. This was not right because I had verified there were no vApps tied to the Network Pool. Furthermore the item listed still in use was a dynamically created dvportgroup which also no longer existed on the vNetwork Distributed Switch in vCenter.

I suspect this situation came about due to running out of available storage space earlier in the week on the Microsoft SQL Server where the vCloud database is hosted. I was performing Network Pool work precisely when that incident occurred and I recall an error message at the time in vCloud Director regarding tempdb.

I tried removing state data from QRTZ tables which I blogged about here a few years ago and has worked for specific instances in the past but unfortunately that was no help here. Searching the VMware Communities turned up sparse conversations about roughly the same problem occurring with Org vDC Networks. In those situations, manually editing the vCloud Director database was required.

An obligatory warning on vCloud database editing. Do as I say, not as I do. Editing the vCloud database should be performed only with the guidance of VMware support. Above all, create a point in time backup of the vCloud database with all vCloud Director cell servers stopped (service vmware-vcd stop). There are a variety of methods in which you can perform this database backup. Use the method that is most familiar to and works for you.

Opening up Microsoft SQL Server Management Studio, there are rows in two different tables which I need to delete to fix this. This has to be done in the correct order or else a REFERENCE constraint conflict occurs in Microsoft SQL Server Management Studio and the statement will be terminated.

So after stopping the vCloud Director services and getting a vcloud database backup…

Step 1: Delete the row referencing the dvportgroup in the [vcloud].[dbo].[network_backing] table:

Step 2: Delete the row referencing the unwanted Network Pool in the [vcloud].[dbo].[network_pool] table:

That should take care of it. Start the vCloud Director service in all cell servers (service vmware-vcd start) and verify the Network Pool has been removed.

VMware Releases vSphere PowerCLI 5.5 R2

March 12th, 2014

I stumbled across some interesting news shared by Alan Renouf on Facebook this morning – an R2 release of vSphere PowerCLI 5.5 (Build 1649237).  New in R2 per the release notes:

  • Access to the vCenter Server SRM public API (Connect-SRMServer and Disconnect-SRMServer cmdlets) – an exciting addition for sure
  • Support for adding and removing tags and tag categories found in the next generation vSphere web client
  • Configuration and reporting of EVC mode for vSphere clusters
  • Management of security policies for the vSS and its portgroups
  • New support for MS Windows PowerShell 4.0
  • Support for vSphere hosts configured for IPv6
  • Added migration priority support for vMotion (VMotionPriority parameter in conjunctionw ith the Move-VM cmdlet)
  • Get-Datastore cmdlet
    • RelatedObject paremeter extended to accept the Harddisk object
    • now allows filtering by cluster
  • Enhanced Get-Stat and Get-StatType cmdlets
  • Support added for e1000e vNICs
  • All values for DiskStorageFormat can be specified during VM cloning operations
  • 64-bit mode support for New-OSCustomizationSpec and Set-OSCustomizationSpec cmdlets
  • ToolsVersion property added to VMGuest which returns a string
  • Get-VirtualSwitch and Get-DVSwitch cmdlets support virtual port groups as a RelatedObject
  • Get-VM cmdlet enhanced to retrieve a list of VMs by virtual switch
  • Miscellaneous bug fixes

VMware vSphere PowerCLI 5.5 R2 supports vSphere 4.1 through vSphere 5.5 as well as Microsoft Windows PowerShell versions 2.0, 3.0, and new in R2 4.0.

Thank you Alan and thank you VMware!

Microsoft Sysprep Change in vCloud Director 5.5

November 18th, 2013

If you’re like me, you still support legacy Windows operating systems from time to time.  Let’s face it, Windows Server 2003 was a great server operating system and will probably remain in some environments for quite a while.  I won’t at all be surprised if the Windows Server 2003 legacy outlasts that of Windows XP.  To that point, even the VCAP5-DCA  exam I sat a few weeks ago used Windows Server 2003 guests in the lab.

All of that being said in what is almost the year 2014, hopefully you are not still deploying Windows Server 2003 as a platform to deliver applications and services in a production environment.  However, if you are and you’re using VMware vCloud Director 5.5, you should be aware of subtle changes which I noticed while reading through the documentation.  Page 31 of the vCloud Director 5.5 Installation and Upgrade Guide to be exact.

In previous versions of vCloud Director including 5.1, Microsoft Sysprep files were placed in a temporary directory within operating system specific folders on the first cloud cell server in the cluster.  The next step was to invoke the /opt/vmware/vcloud-director/deploymentPackageCreator/createSysprepPackage.sh script which bundled all of the Sysprep files into a /opt/vmware/vcloud-director/guestcustomization/windows_deployment_package_sysprep.cab file.  At this point, Sysprep was installed and configured on the first cell server.  It could then optionally be distributed by way of copying the .cab file and the vcloud_sysprep.properties file to the guestcustomization directory of the other cell servers in the cluster.  I call this step optional because not all vCloud deployments will have multiple cell servers.  If multiple cell servers did exist, you’d likely want all of them to be able to perform guest customization duties for legacy Windows operating systems in the catalog and thus this optional step would be required.

So a few things have changed now in 5.5.  First, the Windows operating system specific folder names have changed to match the folder names which vCenter Server has always used historically (see VMware KB 1005593) and on this note, Windows 2000 Server support has been put out to pasture in vCD 5.5.

Version pre-vCD 5.5 vCD 5.5
Windows 2000 /win2000 unsupported
Windows Server 2003 (32-bit) /win2k3 /svr2003
Windows Server 2003 (64-bit) /win2k3_64 /svr2003-64
Windows XP (32-bit) /winxp /xp
Windows XP (64-bit) /winxp_64 /xp-64

Next, the method to create the Sysprep package and distribute it to the other cell servers has changed.  The createSysprepPackage.sh script no longer exists and as a result, a bundled .cab file is not created.  Instead, the Sysprep files are copied in their entirety to their new directory names within the directory /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep.  So what you need to do here is create the directory structure under $VCLOUD_HOME and SCP the Sysprep files to each of the cell servers.  I’ve provided the directory creation commands below:

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/svr2003

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/sv42003-64

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/xp

mkdir -p /opt/vmware/vcloud-director/guestcustomization/default/windows/sysprep/xp-64

As the documentation reminds us, the Sysprep files must be readable by the user vcloud.vcloud (this user is created on each cell server during the initial vCloud Director installation) and that can be ensured by running the following command:

chown -R vcloud.vcloud $VCLOUD_HOME/guestcustomization

These installation changes are important to note if you’re deploying a net new vCloud Director 5.5 environment and there is a need for legacy Windows OS vAPP guest customization.  A vCloud Director 5.5 upgrade from previous versions will perform the necessary Sysprep migration steps automatically.  Lastly, Sysprep won’t be needed in vCloud environments where guest customization isn’t required or legacy versions of Windows aren’t being deployed and customized (Beginning with Windows Vista and Windows Server 2008, Sysprep is bundled within the operating system).