Storage block size and alignment

March 20th, 2009 by jason Leave a reply »

Steve Chambers posted version 2 of the Storage block size and alignment document over at the VIOPS (VMware Virtual Infrastructure Operations) site. At seven pages, it is both a short and a GREAT read.

For those not familiar with VMFS and VM guest alignment, I’ll summarize:

VMFS Alignment

  1. Unaligned volumes result in track crossing and additional I/O penalties in the form of latency and throughput which may or may not be noticeable in your environment (it depends)
  2. To verify whether or not your VMFS volumes are aligned, run the fdisk -lu command at the console
  3. VMFS volumes created with the Virtual Infrastructure Client (vSphere Client) are automatically aligned since it automatically align the volume along the 64KB boundary so no need to worry about the sub bullets in #2 above.
  4. NFS datastores are not concerned with VMFS alignment as they are not block VMFS datastores
  5. Alternatively, VMFS volumes can be aligned by following a series of fdisk commands manually which will destroy data on the volume (definitely not preferred)
  6. VMFS block size only determines maximum file size on the VMFS volume. VMFS block size does not play even a remotely significant performance role.  There are a number of expert blog articles which debate this.

VM Guest Alignment

  1. To verify whether or not your VM guest virtual disks are aligned, check the partition offset value
    • Aligned virtual disks will have a partition offset value evenly divisible by 4,096 (ie. 65,536 or 1,048,576 which is a default for Windows Server 2008)
    • Non-aligned virtual disks will have a partition offset value not evenly divisible by 4,096 (ie. 32,256 which is a default for Windows XP and Server 2003)
  2. Due to the destructive nature of the alignment procedures, alignment is always performed before data is placed on the volume
  3. Alignment in Linux guests is performed using an almost identical series of fdisk commands listed in a previous bullet
  4. Alignment in Windows guests is performed using diskpart.exe
  5. Although guest alignment is data destructive, guest alignment can be performed after the guest OS is installed because the document recommends that alignment of the OS partition is unnecessary; only align the data partitions before data is placed on them.  **see update below**

Alignment is most often going to be labor intensive and thus will have diminishing returns. This will especially be true if your environment has already been built and you need to align after the fact. Environments in the planning stages and not yet built will be among the best candidates for alignment right out of the gate. Whatever stage you are at, updating guest VM templates with alignment wouldn’t be a bad idea. Alignment of one image will pay dividends, whether noticeable or not, over and over as that template is deployed throughout the infrastructure.

Update: NetApp released a few scripts that will not only automate the verification and alignment processes at the guest VM OS level, the script will align the guest OS without destroying data. The one exception I ran into was with a Citrix VM that had remapped drives. CTXGINA.DLL got real cranky. The scripts are:

  • mbrscan – Scans the -flat.vmdk file for alignment
  • mbralign – Makes a backup of the .vmdk and creates a newly aligned .vmdk

See also:  NetApp – Storage Nuts & Bolts: mbrscan/mbralign

3-20-2009 1-24-50 PM

Other recommended reading:

Recommendations for Aligning VMFS Partitions

Performance Best Practices for VMware vSphere 4.1

Advertisement

3 comments

  1. pdonlin says:

    Hi Jason,

    Just to add a note. If your Windows guest OS is Windows Server 2008, you should not need to manually align your partitions because Widows Server 2008 by default sets the offset to 1024KB

    Pete

  2. Windows Vista and above all move to a 1MB offset, but you still need to be careful. I’ve found that about around 1 in 20 still get offset incorrectly, sometimes by as little as 2 bytes. It’s still possible, but you are correct it is less of an issue for the newest versions of Windows.

  3. Dave Holland says:

    Thanks for this useful summary. If you’re not using partitions – e.g. by giving the entire virtual LUN to LVM in the guest machine, so there is no partition table – are there still alignment issues in the guest/VMFS layer? (I’m assuming VMFS/LUN is aligned, because VI Client gets it right.)

Leave a Reply