The Lab – Then and Now

Documenting its evolution using VMware

The lab is a critical component in my my learning and memory retention process. Most of my training and certification has been acquired with the benefit of using the lab. The time and money used in building, maintaining, and cooling the lab I believe has been money well spent in terms of career opportunities and advancement.

At its beginning, the lab consisted of every piece of hardware I had physical space, electrical, and network ports for. The more the merrier. However, underlying problems were developing that, ironically, some of today’s businesses still deal with. There was very little consistency and most of the hardware was what I would call low end or desktop class. The hardware didn’t stack well vertically, was failure prone, wasn’t very scalable, and had no redundancy. Although it was fun for quite some to set up, tear down, etc., I had been through enough of it and my days of A+ certification were far behind me. I was starting to experiment with a lot of different operating systems and configurations. This usually meant a fresh, brand new installation of an OS on a piece of hardware each time. As I grew more curious about the software and OS side of things, more OS builds were needed. Since each OS was installed on dedicated hardware, there existed a 1:1 relationship between running OS experiments and physical hardware in the lab (and electrical, and network, and tangled cords, etc.). I was getting a bit tired of the mess of hardware I was dealing with.

Then I was introduced to VMware. Chances are if you are reading this, you have an idea of what VMware was able to do for the lab. Eliminate hardware. Reduce hardware complexity. Introduce virtual hardware platform consistency. Flexibility. Uptime. Efficiency. Compatibility. Etc. I’ll be honest in saying that VMware doesn’t solve all of the labs problems. It eliminated most of the old problems, but at the same time introduces a few new challenges. Instead of many small boxes, I now have a few large ones. The main obstacle is cost. This isn’t to say that VMware makes virtualization more expensive. Quite the opposite in most cases. But now instead of using free hand-me-down computers in the lab, I need to pony up money for decent hardware with server class (and sometimes datacenter class) operational characteristics: performance, scalability, redundancy, fault tolerance, vertical stack, etc. I welcome these challenges though as they get me thinking deeper and more strategically (good exercise for my career brain), plus they bring my lab that much more in alignment with what I work with on a daily basis in my career.

Lab Update 1/19/10:

I thought I’d post a lab update since John Troyer nudged me letting me know this week’s weekly podcast was focusing home labs for VCP and VCDX studies.  My lab has grown to what some may consider a ridiculous size.  I’ve received comments such as “Why do you need all that stuff for your home?”  My response is “that stuff” is not for my home, it’s for my lab, which happens to be in my home.  If you want to talk about what’s ridiculous, have a look at co-location costs.  I’ve been monitoring them for years looking for something affordable.  Thus far I can’t justify the rent they charge, plus the access to equipment is an inconvenient 30 minute drive, very limited, and fee-based.  That’s why the lab is still in my home.  No I’m not going to lab in the cloud.  I need my hands on equipment which clouds won’t give me such as fibre channel switches and SANs.  What’s important is that the lab suits me and my career pursuits well. 

With the exception of a physical file server which will be virtualized soon, the lab is used 100% for VMware virtualization.  The DL385 G2 hardware is deployed as vSphere (FT compatible) hosts.  These make nice virtual infrastructure hosts with plenty of power to run many VMs including additional ESX hosts deployed as VMs for testing various things.  For quite some time now I’ve been running the pair in a two-host cluster, one as an ESX host and the other as an ESXi host.  This may seem odd but VMware does support the mixed configuration.  The reason for the mix is that I need to stay up to speed on both platforms.  A mixed cluster is a good way to do that and it has actually uncovered an incompatibility issue which I was able to report to VMware.

The lab has seen a few exciting additions in the past month.  I recently picked up an EMC Celerra NS-120 Unified Storage SAN which added a second rack to my lab as well as two 220V 30 Amp Single Phase circuits to provide the power to it.  An additional HP StorageWorks 2/8V fibre channel switch should arrive tomorrow to complete a pair for more in depth NPIV testing and to extend the SAN fabric from the HP MSA1000 SAN to the EMC Celerra NS-120 SAN.  Arriving soon will be NetApp FAS3000 Series storage.  With the new EMC and NetApp storage, I should be able to retire the HP MSA1000 SAN.  It’s a decent SAN but it’s pretty old, slow’ish, and its a one trick FC pony - not nearly as full featured as the EMC and NetApp offerings.  I’m looking forward to things like Dedupe (3D), Thin Provisioning, File Level Retention (FLR), Replication, etc.  Finally, a pair of Emulex dual port fibre channel PCI-Express HBAs which support NPIV.

Here’s a snapshot of the lab today:

Rack 1 (three 110V 20 Amp circuits)

• NetGear JGS524 24-port Gb Ethernet switch
• Cisco Catalyst 3500 XL 24-port Fast Ethernet switch
• HP StorageWorks 2/8V SAN switch
• HP StorageWorks 2/8V SAN switch
• HP DL385 (1x AMD DC Opteron, 4GB RAM)
• HP MSL5026 fibre channel tape library
• 15″ LCD rack monitor
• HP DL385 (1x AMD DC Opteron, 4GB RAM)
• HP DL380 G3 (2x Intel P4 XEON, 6GB RAM)
• KVM switchbox/Keyboard tray
• HP DL385 G2 (2x AMD QC Opteron, 16GB RAM)
• HP DL385 G2 (2x AMD QC Opteron, 16GB RAM)
• APC SMART UPS 2200
• APC SMART UPS 2200

Rack 2 (two 220V 30 Amp single phase circuits)

• EMC Celerra NS-120 Unified Storage SAN (45 spindles, 6.5TB raw)
• NetApp FAS3050c Multiprotocol SAN (2 heads, 42 spindles, 7.5TB raw)