Posts Tagged ‘Networking’

VMkernel Networks, Jumbo Frames, and ESXi 4

February 12th, 2010

Question:  Can I implement jumbo frames on ESXi 4 Update 1 VMkernel networks?

Answer:  Who in the hell knows?

You see, the ESXi 4.0 Update 1 Configuration Guide states on page 54:

“Jumbo frames are not supported for VMkernel networking interfaces in ESXi.”

Duncan Epping of Yellow Bricks also reports:

“Jumbo frames are not supported for VMkernel networking interfaces in ESXi. (page 54)”

One month after the release of ESXi 4 Update 1, Charu Chaubal of VMware posted on the ESXi Chronicles blog:

“I am happy to say that this is merely an error in the documentation. In fact, ESXi 4.0 DOES support Jumbo Frames on VMkernel networking interfaces. The correction will hopefully appear in a new release of the documentation, but in the meantime, go ahead and configure Jumbo frames for your ESXi 4.0 hosts.”

Shortly after, Duncan Epping of Yellow Bricks confirms Charu Chaubal’s report that jumbo frames are supported on ESXi VMkernel networks.

Now, nearly two months after Charu’s clarification and three months after the release of ESXi 4 Update 1, the documentation remains dubious on page 54 stating that jumbo frames are not supported on ESXi 4 VMkernel networks which is a direct contradition to a VMware ESXi blog.

I opened a Business Critical Support SR with VMware on the question.  I was told by VMware BCS that jumbo frames are NOT supported on ESXi 4 Update 1 VMkernel networks and a reference was made to the documentatation on page 54. 

Our dedicated VMware onsite Engineer escalated and I was then told ESXi 4 Update 1 DOES support jumbo frames on VMkernel networks, making reference to Charu’s article.

Hey VMware, which is it?  If this is a documentation mistake, why are you dragging your feet in getting the documentation updated two months after a VMware employee discovers the error and blogs it?  Waiting for the next release of ESXi?  Unacceptable!  You update the public documentation as soon as you discover the error and be damned sure your BCS support Engineers know the right answer!  Do you know how much companies pay for BCS?  You owe your customers the correct answer.  If misinformation comes as a result of a known documentation error, SHAME ON YOU!  Architecture and design decisions are being made daily on this information or misinformation, which ever it may be.

Update 2/23/10:  Toby Kraft (@vmwarewriter on Twitter) will be updating the documentation by next week.  Thank you Toby!

Update 3/1/10:  VMware has updated their documentation to reflect currently supported configurations.  Thank you VMware (and Toby)!

Virtualizing vCenter With vDS Catch-22

October 9th, 2009

I’ve typically been a fan of virtualizing the vCenter management server in most situations. VMware vCenter and Update Manager both make fine virtualization candidates as long as the underlying infrastructure for vCenter stays up. Loss of vCenter in a blackout situation can make things a bit of a hassle, but one can work through it with the right combination of patience and knowledge.

A few nights ago I had decided to migrate my vCenter VM to my vSphere virtual infrastructure. Because my vCenter VM was on a standalone VMware Server 2.0 box, I had to shut down the vCenter VM in order to cold migrate it to one of the ESX4 hosts directly, transfer the files to the SAN, upgrade virtual hardware, etc. Once the files were migrated to the vSphere infrastructure, it was time to configure the VM for the correct network and power it up. This is where I ran into the problem.

vCenter was shut down and unavailable, therefore, I had connected my vSphere client directly to the ESX4 host in which I transferred the VM to. When trying to configure the vCenter VM to use the vNetwork Distributed Switch (vDS) port group I had set up for all VM traffic, it was unavailable in the dropdown list of networks. The vCenter server was powered down and thus the vDS Control Plane was unavailable, eliminating my view of vDS networks.

This is a dilemma. Without a network connection, the vCenter server will not be able to communicate with the back end SQL database on a different box running SQL. This will cause the vCenter server services to not start and thus I’ll never have visibility to the vDS. Fortunately I have a fairly flat network in the lab with just a few subnets. I was able to create a temporary vSwitch and port group locally on the ESX4 host which would grant the vCenter VM the network connectivity it needed so I could then modify the network, changing from a local to a vDS port group on the fly.

Once the vCenter server was back up, I further realized that vDS port groups are still unable to be seen when the vSphere client is connected directly to an ESX4 host. The ability configure a VM to utilize vDS networking requires both that the vCenter server be functional, as well as a vSphere client connected to said vCenter server and not a managed host.

The situation I explained above is the catch-22 – the temporary inability to configure VMs for vDS networking while the vCenter server is unavailable. One might call my situation a convergence of circumstances, but with an existing virtualized vCenter server that you’re looking to migrate to a vDS integrated vSphere infrastructure, the scenario is very real. I’d like to note all VMs that had been running on a vDS port continued to run without a network outage as the vDS Data Plane is maintained on each host and remained in tact.

Lab Manager 4 and vDS

September 19th, 2009

VMware Lab Manager 4 enables new functionality in that fenced configurations can now span ESX(i) hosts by leveraging vNetwork Distributed Switch (vDS) technology which is a new feature in VMware vSphere. Before getting overly excited, remember that vDS is a VMware Enterprise Plus feature only and it’s only found in vSphere. Without vSphere and VMware’s top tier license, vDS cannot be implemented and thus you wouldn’t be able to enable fenced Lab Manager 4 configurations to span hosts.

Host Spanning is enabled by default when a Lab Manager 4 host is prepared as indicated by the green check marks below:

When Host Spanning is enabled, an unmanageable Lab Manager service VM is pinned to each host where Host Spanning is enabled. This Lab Manager service VM cannot be powered down, suspended, VMotioned, etc.:

One ill side effect of this new Host Spanning technology is that an ESX(i) host will not enter maintenance mode while Host Spanning is enabled. For those new to Lab Manager 4, the cause may not be so obvious and it can lead to much frustration. An unmanageable Lab Manager service VM is pinned to each host where Host Spanning is enabled and a running VM will prevent a host from entering maintenance mode. Maintenance mode will hang at the infamous 2% complete status:

The resolution is to first cancel the maintenance mode request. Then, manually disable host spanning in the Lab Manager host configuration property sheet by unchecking the box. Notice the highlighted message in pink telling us that Host Spanning must be disabled in order for the host to enter standby or maintenance mode. Unpreparing the host will also accomplish the goal of removing the service VM but this is much more drastic and should only be done if no other Lab Manager VMs are running on the host:

After reconfiguring the Lab Manager 4 host as described above, vSphere Client Recent Tasks shows the service VM is powered off and then removed by the Lab Manager service account:

At this time, invoke the maintenance mode request and the host will now be able to migrate all VMs off and successfully enter maintenance mode.

While Lab Manager 4 Host Spanning is a step in the right direction for more flexible load distribution across hosts in a Lab Manager 4 cluster, I find the process for entering maintenance mode counter intuitive, cumbersome, and at the beginning when I didn’t know what was going on, frustrating. Unsuccessful maintenance mode attempts have always been somewhat mysterious in the past because vCenter Server doesn’t give us much information to pinpoint the problem as far as what’s preventing the maintenance mode. This situation now adds another element to the complexity. VMware should have enough intelligence to disable Host Spanning for us in the event of a maintenance mode request, or at the very least, tell us to shut it off since it is conveniently and secretly enabled by default during host preparation. Of course, all of this information is available in the Lab Manager documentation, but who reads that, right? :)

Thank you Gabe and Brenda

September 14th, 2009

I’d like to take a moment to thank two people, Gabe and Brenda, for their new and continuous friendship. They hail from the Netherlands and the pair are two of the nicest, funniest, and fun loving people I’ve met. I was first introduced to them in person earlier this year in Cannes, France during the VMworld Europe 2009 virtualization conference. Gabe was attending as a VMware user and Brenda joined him to study conference attendees in their preferred habitat, as well as for some sight seeing. Being from the U.S., I was quite out of my environment while traveling for the first time in France but they made me feel welcomed, teaching me some of the local customs as well as bits and pieces of the French language: “Merci beaucoup” – “Thank you very much” – a valuable phrase for a clueless tourist to individually thank each person for their assistance.

I met up with them again at VMworld 2009 in San Francisco, CA. This time they treated myself, my wife, and my kids to a nice Italian dinner Thursday evening after the conclusion of the conference. In addition, they showered my children with authentic Dutch gifts. Gabe and Brenda, if you are reading this, we very much appreciated this – Thank You! I hope one day we will meet again so that I can reciprocate. Chances are good as I’ve mentally committed to attend at least one VMworld annually, expending whatever efforts it takes to get there.

Where can you find this dynamic duo?

Brenda maintains a very interesting blog called Virtual Gipsy which offers an Anthropologist’s perspective of a tight knit virtualization community. Follow her on Twitter: @b_renda

Gabe runs an excellent virtualization blog called Gabe’s Virtual World and is particularly good with video editing. Follow him on Twitter: @gabvirtualworld

Lab Manager Network Port Requirements

May 13th, 2009

I need to become a VMware Lab Manager expert and so it begins.  From what I’ve seen so far, Lab Manager 3.x has made great progress since I last kicked the tires 15 months ago on Lab Manager 2.x.  The biggest news by far is that ESX hosts can be managed both by Lab Manager Server and vCenter Server with all the fixins (DRS, HA, VMotion).  Although I’ve already found that VMs connected to an internal only vSwitch remain pinned to the host due to VMotion rules.

Nothing too Earth shattering here; this information comes straight from page 20 of the Lab Manager Installation and Upgrade Guide.

Systems TCP Port UDP Port
Client browser to access Lab Manager Server system 443
Client browser to access ESX hosts 902, 903
Lab Manager Server system and ESX hosts to access SMB share

(import and export operations only)

139, 445 137, 138
ESX hosts to access NFS media datastores or NFS virtual machine datastores 2049
Lab Manager Server system to access Lab Manager agent on ESX hosts 5212
Lab Manager Server system to access ESX host agent on ESX hosts 443
Lab Manager Server system to access the VirtualCenter Server system 443
Lab Manager Server system to communicate with virtual router on some ESX hosts

(for fenced configurations)

514
Lab Manager Server system to access LDAP Server 389 LDAP

636 LDAPS

Before the installation of Lab Manager, be sure that ports above won’t conflict with an existing configuration by running the netstat -b command from the Windows command line.

Cloud Camp Minneapolis

April 18th, 2009

IMG00028-20090418-1006Today I attended Cloud Camp Minneapolis from 9:00am to 3:30pm on the University of Minnesota East Bank campus. I think the event was large success. Registration was SOLD OUT and it looked like there was somewhere between 100 and 150 attendees. I think it speaks well for the technology and the event organization when that many people will give up the majority of an absolutely gorgeous Saturday.

The event started with a continental like breakfast where people mingled and socialized for an hour before the speaking agenda began. I ran into a few familiar faces and also met with new people I hadn’t met before. The coffee was strong and the bagels looked good.

After breakfast, we were ushered into the main auditorium. George Reese (pictured top left), cloud book author and event organizer from enStratus Networks, kicked things off by briefly introducing himself as well as the premier sponsors: VISI, enStratus, Microsoft, Hosso The Rackspace Cloud, Aserver, and Right Scale.

Shortly after, the Lightning Talks began. This is where the premier event sponsors were allowed just a few minutes to deliver their cloud speech along with a little product marketing while literally whipping through their slide deck. When I say just a few minutes, I literally mean it. I think five vendors all got up and delivered their presentations in a total of 15 minutes. If you’ve ever watched the television program “Mad Money”, it was like cloud talk and offerings during the lightning round. It was both an interesting and refreshing approach.

Next we had a lengthy group discussion on hot cloud topics which were in turn used to dynamically develop the afternoon breakout session topics. We touched on things such as security, mobility, legal and liability implications, small business, etc.

We broke for lunch where I had discussions with a few locals on phone, cable, and internet service providers (ISPs) in the state of Minnesota.

After lunch the large group broke up into the smaller breakout sessions mentioned previously. I attended two sessions: Mobility and SMB.

The mobility session had a good crowd mixture comprised of service providers, application developers, and CEOs. The discussions jumped from topic to topic as people offered up their problems, questions, and philosophies orbiting cloud mobility and isolation. Not to my surprise, there was very little along the lines of answers or solutions. That’s ok. I wasn’t expecting any. Frankly, I found comfort among large numbers of industry experts who, like I, didn’t have the answers and were just as perplexed about figuring out how this is all going to work out. Developers seemed to be the most concerned about the application layer (Applications as a Service) as discussions touched on APIs and applications in the cloud and their impact on development techniques as it applies to mobility. I got a sense of less concern over platform in the cloud, also known as Platform as a Service. One developer talked about his current experience of using Amazon’s Elastic Compute Cloud (EC2). His direct benefits: he owns and supports nothing, and he pays only for what he uses. When he’s not using it, there’s essentially little or no cost. When he’s done, I imagine he saves what he needs, and the rest is destroyed. There is no traditional decommissioning and writing off of assets. There is no hardware that needs to be disposed of properly.

The SMB session was another good mixture of attendees nearly the same as above but with more of a concentration on small business, as well as micro and nano business (phrases coined during the session representing entities smaller than small business). The general idea of this session was if and how small businesses can benefit from cloud offerings. Talks began with the various ways to define a small business: by revenue? by headcount? by technology? There are examples of large manufacturing plants that have small technology footprints. Likewise, small operations can generate large amounts of revenue with the assistance of technology. Group members proposed that there exists many inefficiencies in small business, particularly in the technology and infrastructure. This is where renting platforms, applications, services, and infrastructure from cloud providers could make sense for SMBs. Wouldn’t small businesses rather focus their time and energy on developing their products and services instead of being tied down by the technology they need to run their business on? From a customer or partner credibility standpoint, does a business look more professional and equipped running their business in a certified cloud datacenter, or a broom closet? What impacts will regulation and legislation have? Decisions of how to securely store and deliver customer information in a small business shouldn’t be taken lightly. There are consequences that could easily break the trust and financial backing that a small business or startup’s survivability relies on.

In all, I had a great time at Cloud Camp Minneapolis. If you asked me six months ago what I know about the cloud, I would have had nothing to say other than “I don’t get it”. I’ve gradually been warming up to the concept and today Cloud Camp Minneapolis went a long way in delivering my first feeling of personal and professional accomplishment in that I think I’m actually caught up and on the same page as many of my peers and higher experts in the cloud community. However, I have to be honest in saying that I walked away somewhat disappointed and in disbelief that virtualization discussion was nearly non-existent. The last two VMworld virtualization conferences I attended in Las Vegas and Cannes were strongly focused on cloud computing and VMware’s Virtual Datacenter OS (VDC-OS). There was maybe one mention of VMware in one sentence and a brief reference to VDI. Microsoft was on site talking about Azure and there was no mention of Hyper-V. No mention of XenServer, Virtual Iron, etc. I’ve been led to understand that virtualization is key component to cloud infrastructure, applications, and mobility. I anticipated much of today’s discussions would revolve around virtuailzation. I couldn’t have been more wrong. After the event finished, I sent out a tweet re: no virtualization talk today. I received a response stating virtualization is merely a widget or one small component among many in the cloud. Virtualization is not really as integral as I’m being told by Paul Maritz of VMware. Maybe this is a case of Jason has been drinking too much VMware Kool-Aid for too long. The answers about the cloud are coming. Slowly but surely. Hopefully Paul is right and VMware does have a significant role to play in their version of global cloud computing. I’d like to see it, realize it, and experience it.

Twitter outage

February 19th, 2009

I don’t see this often (in fact I’ve never seen this) so I thought it was blog worthy given the increased use of social media as a professional’s tool.

2-19-2009 12-15-10 AM

Where to get timely VMware virtualization information

December 25th, 2008

Happy Holidays!  I thought tonight was the night I was going to post some “Citrix XenApp virtualized on VMware ESX” that many have been asking me for behind the scenes, but alas it’s 10:30pm and I just don’t have the energy for such a post that will require considerable effort to put together.  I’ve accumulated some information here and there for various people, but it’s time to formally consolidate the scattered pieces of information into one decent post that I can fine tune as needed going forward.  Before you start licking your chops in anticipation of a rocket science blog post on virtualizing Citrix, please don’t.  What I promise is the details and discoveries behind one person’s virtualized Citrix environment.  With VI3, virtualizing Citrix is fairly straightforward but extra special attention must be paid in determining virtualization candidacy.

Now I wouldn’t want anyone to walk away empty handed from my blog on Christmas so I leave you with this:  A no-frills post revealing the source of where I get 90-95% of my daily virtualization information – RSS feeds of various blogs and websites.  This file (right click, save as – it’s XML) contains an export of all of my RSS subscriptions.  Import it into your favorite RSS reader.  Set your RSS subscription refresh interval to 15 minutes.  Stay informed with nearly up to the minute and late breaking VMware virtualization news.  With new blogs and sites popping up weekly, for sure this list is nowhere near what I would call complete.  If you have any suggestions or if you see a great blog or site that I am missing, by all means, let me know in the comment section below.  I’m the type of guy that can never get enough VMware virtualization information.

Disclaimer:  My RSS subscription list contains a few subscriptions to non-virtualization related feeds which you may want to remove.

Update:  I’ve added two more great blogs to the RSS feeds:  Gabe’s Virtual World (Gabrie van Zanten) and Jase’s Place (Jason McCarty).

Further unwrapping of the free tool from Veeam

December 18th, 2008

Rich Brambley of VM /ETC allowed me to take a look at the Veeam present located under his tree.  Due to our carelessness, more wrapping paper seems to have been worn away!

Can anyone guess what this tool might be?  I’ve got a hunch and my guess can be found in the form of a tag below this blog entry.

Be sure to register for free copy of this tool being made available by Veeam on December 22nd!

Happy Holidays!

12-18-2008 1-37-13 PM

New VMware VI network port diagram request for comments

December 12th, 2008

Quick update I’ve been meaning to post for a few weeks now – sorry for the delay.  I received a new network diagram that reader Shlomo Rivkin has been working on and he would like some community input on it.  Here’s the new version being submitted for discussion:

12-12-2008 5-04-56 PM

The high res version of the above diagram is here.

Feel free to compare and contrast it to the version below which is posted on my blog as well as the VMware VMTN communities:

vmware_network_ports

The high res version of the above diagram is here.

Sorry for the shortness of this post – heading to a parade with my family.

Maintenance tonight

December 9th, 2008

The blog, web, and Team Fortress 2 servers will be down briefly tonight for a little maintenance on the virtualized gateway router.  Duration should be about half an hour at the most.  I apologize in advance for any inconvenience.

Speaking of maintenance, I doubled my hosting bandwidth over the weekend from 5Mbps down/512Kbps up to 10Mbps down/1Mbps up.  I performed a little bandwidth speed testing last night and initially I wasn’t overly pleased the results.  Depending on the remote host I tested speed against, I wasn’t seeing the numbers I should be on the download side.  Eventually I did find a remote host that proved I had a 10Mbps down pipe (I don’t have bursting AFAIK).  On the up side (which is what really counts for hosting performance and you readers), I wasn’t able to find any remote hosts that showed I had upstream bandwidth beyond 512Kbps.  I’ll be performing more tests and I will contact my service provider if I am not completely satisfied.  For what I’m paying for business class broadband, I insist that I be consistently getting the 80% of the promised speeds which I believe is the SLA with my provider.

Trust me, I could go really hysterical with regards to my provider but you readers deserve better so I’ll keep it bottled up for now.  Thank your lucky stars for whatever provider you have because chances are they are much better than what I have to work with.

Toodles.

Update: Bandwidth is looking good.  Explanation in comments below.

12-9-2008 9-45-07 PM

Build a network boot disk for VMware guest VMs

November 25th, 2008

A person recently asked me via Email how to create a bootable MS-DOS diskette with networking capability for use in VMware guest VMs. Rather than privately isolate the knowledge in an email conversation, I figured the least I could do after going through the steps is to share it in a blog post so that it may be cataloged in Google for everyone’s benefit.

There are several methods to creating a network boot disk. Some easier. Some more difficult. In the interest of time and leveraging the innovation of others, I’ll turbo charge today’s procedure by using Bart’s Network Boot Disk. Frankly I’m not interested in modifying network boot disk files by hand which was one of the purposes behind Bart’s solution – making the creation of boot disks easier. Note, to use this procedure, you admit to owning a Microsoft Windows 98 operating system license.

Here are the steps:

  1. Create the boot disk by following the instructions here.
  2. Download the BFD full package v1.0.7 file.
  3. Extract bfd107.zip to a temporary folder (I’ll use c:\temp\ for this example).
  4. Good news – the driver used by VMware (the AMD PCNet Family Ethernet Adapter NDIS pcntnd.cab) is already included in the default list of drivers bundled in the bfd107.zip file above. This is a perfect working example of why VMware chose to virtualize the AMD PCNet Family adapter. It’s ubiquitous nature allows it to be supported by every VMware guest operating system on the support list. By virtue of the fact that VMware supports most of the popular/common Windows and Linux operating systems, you’ll find that VMware networking works with nifty utilities like Bart right out of the box.
  5. As the instructions indicate, open a command prompt, go to the BFD directory (in this example, c:\temp\) and execute the command bfd msnet and follow the instructions on screen. This step will create the actual floppy diskette.
  6. The network boot diskette is ready to use with VMware. Use it to boot a guest VM.
  7. I found that booting from the #3 menu item labeled “Boot without emm386″ worked well with ESX 3.5.0 Update 3:
    11-25-2008 4-59-37 PM
  8. Accept the following default prompts assuming they are applicable to your environment:
    11-25-2008 5-05-03 PM 11-25-2008 5-05-40 PM 11-25-2008 5-05-49 PM 11-25-2008 5-05-57 PM
  9. Configure the “Logon as”, “Password”, “Workgroup”, and “Domain” as necessary:
    11-25-2008 5-06-32 PM
  10. The network boot disk will complete its boot up process, connecting your MS-DOS VM to the network with the given parameters. A quick net view displays the shares of a Windows server on the network:
    11-25-2008 5-08-17 PM
  11. A net use command maps a C: network drive to the network Windows server share and a dir command displays the share contents:
    11-25-2008 5-08-46 PM

Well that’s about it. At this point, you’re on the network, ready to dump or capture an image, or whatever it is that you needed a network boot disk for. Don’t forget you can transform the physical floppy diskette into a virtual floppy image by using a utility such as WinImage by Gilles Vollant. This allows the VM to boot much more quickly and it allows you to avoid the use of the dying technology of physical floppy disks altogether.

Update: Roger Lund posted another method on his blog using the Universal TCP/IP Network Bootdisk that looks just as quick and easy.  Check out Roger’s solution.

VMware VI network communications and port usage diagram

November 6th, 2008

Nigel Metheringham has taken the information I posted here and updated it with information from page 179 of the VMware ESX Server 3 Configuration Guide

The result is a current diagram of VMware Virtual Infrastructure network communications and port usage which applies to both ESX and ESXi.  Nigel sent me the updated document via email so that I can update the information on the VMware Communities, however, they are currently unavailable due to planned maintenance so in the mean time I’m making the document available here.  Thank you Nigel!

Download from the link below:

vmware_network_ports

Monday morning funny

November 3rd, 2008

I saw this comment on Twitter this morning from @jbjon and it made me  chuckle:

twitter is just multiplayer notepad”

I used to be the person that knocked Twitter, but I have gotten some practical use out of it since joining for VMworld 2008.

I’m still not down with the Myspace and Facebook sites.  Nothing good can come from those sites if you ask me.

Force a DHCP lease renewal in ESX and ESXi

October 31st, 2008

If you are running your ESX or ESXi management console on DHCP and need to force a DHCP lease renewal, here is how it can be done in both ESX and ESXi.

ESX:  Run the following two commands locally in the service console (COS):

ifdown vswif0
ifup vswif0

ESXi:  Use the local console menu to “Restart Management Network”:

esxidhcprenw