Archive for June, 2011

IBM x3850 NICs Lose Network Connectivity With ESXi 4.0 Update 1

June 11th, 2011

This is a heads up on an issue I ran into some time ago upgrading to VMware ESXi 4.0 Update 1 on an IBM System x3850.  Granted, it’s an aging hardware platform and fast becoming a dated issue, nonetheless this information may help someone out of a late night or weekend fiasco.

Shortly after the upgrade, VMs began experiencing intermittent losses in network connectivity.  Tied to the problem, the following error was revealed in the ESXi log files:

WARNING: LinNet: netdev_watchdog: NETDEV WATCHDOG: vmnic7: transmit timed out

The root cause turned out to be a known issue with the e1000e driver on ESXi 4.0u1 and the IBM x3850.  The issue is documented well in VMware KB Article 1010313 (Intel 82571 NICs intermittently lose connectivity with ESX 4.x).  The KB article was updated last April and appears to still be giving VMware fits as it has spread to vSphere 4.1.  According to the KB article:

This issue may occur if the Message Signaled Interrupt (MSI) mode is enabled for the e1000e driver and this mode is not supported in a server platform. This driver supports these three interrupt modes:

  • 0.Legacy
  • 1.MSI
  • 2.MSI-X

ESX 4.0 added support for Message Signaled Interrupts in network and storage drivers. The default interrupt mode for the e1000e driver under ESX 4.x is MSI (1).

The workaround according to the KB article is to configure the e1000e driver to use Legacy (0) Interrupt mode (thus disabling MSI mode) by performing the following:

  1. Open a console to the ESX or ESXi host.
  2. To configure the e1000e module option IntMode and use Legacy (0) interrupts for a 4-port NIC, run the command:
    • esxcfg-module -s IntMode=0,0,0,0 e1000e
    • Note: A mode number must be specified for each NIC port. In case of 2 quad port NICs, specify the mode 0 for all 8 ports with the command:esxcfg-module -s IntMode=0,0,0,0,0,0,0,0 e1000e
  3. On ESX host, run this command to rebuild initrd:
    • esxcfg-boot -b
    • Note: This step is not applicable to ESXi hosts.
  4. Reboot the ESX/ESXi host for the changes to take effect.

Scripted Removal Of Non-present Hardware After A P2V

June 11th, 2011

After converting a physical machine to a virtual machine, it is considered a best practice to remove unneeded applications, software, services, and device drivers which were tied to the physical machine but no longer applicable to the present day virtual machine.  Performing this task from time to time manually isn’t too bad but at large scale, a manual process becomes inefficient.  There are tools available which will automate the process of removing unneeded device drivers (sometimes referred to as ghost hardware).  A former colleage put together a scripted solution for Windows VMs which I’m sharing here. 

Copy the .zip file to the virtual machine local hard drive, extract it, and follow the instructions in the readme.txt file.  I have not thoroughly tested the tool.  No warranties – use at your own risk.  I would suggest using it on a test machine first to become comfortable with the process before using it on production machines or using on a large scale basis.

Download: remnonpresent.zip (719KB)

VMunderground BPaaS (Beantown Party as a Service)

June 3rd, 2011

Event:  VMunderground BPaaS (Beantown Party as a Service): Tech Field Day 6 Edition

Calling all New England VMUG members, guests, and potential sponsors, as of Friday evening, there are still plenty of tickets available for the upcoming VMunderground BPaaS (Beantown Party as a Service).  I’ll be there and I hope to see you there as well.  Following are the announcement details from VMunderground:

VMunderground is happy to announce that we’ll be helping throw a virtualizaiton community bash with the Tech Field Day 6 crew!!  If you are in the Boston area or local enough to Fenway, mark your calendars for Thursday night, June 9.  It will be the mother of all FCoTR user group meetings!!  Planning a dance fight for FCoTR supporters versus the FCoAC (Avian Carrier) supporters.  It will be spectacular!

Seriously, it’s on. It’s happening. And their will be a big giant green monster involved.  Tech Field Day has the EMC Club at historic Fenway Park in Boston and TFD is opening up the event attendence to VMunderground! We have around 100 seats we’re looking to fill.  If you’re in the Boston area this night clear your calendar and sign up for a ticket and get ready to bring your “A” game to the VMunderground BPaaS (Beantown Party as a Service).   

If you’ve had the fortune of attending a VMunderground WuPaaS (Warm-up Party as a Service) at the last two VMworlds, you know that VMunderground is a blast.  Some of the smartest virtualization minds in the world, best sponsors, and always some good food, brews and conversations.  We’re currently rounding up some great sponsors to help support this community focused event. Stay tuned to learn more about this awesome opportunity to rub elbows with Boston’s finest vPeople and the Tech Field Day crew.  

If you’re interested in sponsoring the first installment of BPaaS, please contact Theron Conrey and Sean Clark at beantown@vkaboom.com (parent company of VMunderground).  We have a few slots open for sponsorship, but speak up soon to reserve your spot.

ATTENDEES:  Please note, this is NOT the VMworld party.  This is a first-time joint collaboration with Tech Field Day and you should probably live within Boston area before committing to this event.  VMunderground rocks, but not maybe not enough to fly into Boston for.  😉

TICKETS: Tickets go on sale in two 50 ticket batches: Friday, May 27 at 4 PM Eastern time.  And the evening of Monday, June 6.  There will be 100 tickets.  When they’re gone, they’re gone. Watch twitter or this site for more info.  @vSeanClark and @TheronConrey are a good place to start.