If you have a vSphere 5.0 environment backed by a storage array (SAN) which supports multipathing over two or more active front end ports (or if you have an array with ALUA support), you may be interested in using VMware’s Round Robin PSP (Path Selection Policy) to distribute storage I/O evenly across multiple fabrics and/or fabric paths. One of the benefits with the Round Robin PSP is that it performs the I/O balancing automatically as opposed to manually tuning fabric and path utilization which is associated with the Fixed PSP – typically the default for active/active arrays. If you’re familiar with Round Robin, you’re probably already aware that you can manually change the PSP using the vSphere Client. However, this can become a tedious affair yielding inconsistent configurations since each LUN on each host in the cluster needs to be configured.
A better solution would be to modify the default PSP for your SATP (Storage Array Type Plugin) so that each new LUN presented to the hosts is automatically configured for Round Robin.
Taking a look at the default PSP for each SATP, I see there is a mix of two different PSPs: VMW_PSP_FIXED (generally for active/active arrays) and VMW_PSP_MRU (generally for active/passive arrays). Notice the Round Robin policy VMW_PSP_RR is not the default for any SATP:
[root@lando /]# esxcli storage nmp satp list
Name Default PSP Description
——————- ————- ——————————————————-
VMW_SATP_ALUA_CX VMW_PSP_FIXED Supports EMC CX that use the ALUA protocol
VMW_SATP_ALUA VMW_PSP_MRU Supports non-specific arrays that use the ALUA protocol
VMW_SATP_MSA VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AP VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_SVC VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_EQL VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_INV VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_EVA VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_SYMM VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_CX VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_LSI VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AA VMW_PSP_FIXED Supports non-specific active/active arrays
VMW_SATP_LOCAL VMW_PSP_FIXED Supports direct attached devices
Modifying the PSP is achieved with a single command on each ESXi host (no reboot required):
[root@lando /]# esxcli storage nmp satp set -s VMW_SATP_ALUA_CX -P VMW_PSP_RR
Default PSP for VMW_SATP_ALUA_CX is now VMW_PSP_RR
If I take a look at the the default PSP for each SATP, I can see the top one has changed from VMW_PSP_FIXED to VMW_PSP_RR:
[root@lando /]# esxcli storage nmp satp list
Name Default PSP Description
——————- ————- ——————————————————-
VMW_SATP_ALUA_CX VMW_PSP_RR Supports EMC CX that use the ALUA protocol
VMW_SATP_ALUA VMW_PSP_MRU Supports non-specific arrays that use the ALUA protocol
VMW_SATP_CX VMW_PSP_MRU Supports EMC CX that do not use the ALUA protocol
VMW_SATP_MSA VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AP VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_SVC VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_EQL VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_INV VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_EVA VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_SYMM VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_LSI VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AA VMW_PSP_FIXED Supports non-specific active/active arrays
VMW_SATP_LOCAL VMW_PSP_FIXED Supports direct attached devices
Now when I present a new LUN to the host which uses the VMW_SATP_ALUA_CX SATP, instead of using the old PSP default of VMW_PSP_FIXED, it applies the new default PSP which is VMW_PSP_RR (Round Robin):
To clarify just a little further, what I’ve done is change the default PSP for just one SATP. If I had other active/active or ALUA arrays which used a different SATP, I’d need to modify the default PSP for those corresponding SATPs as well.
This is good VCAP-DCA fodder. For more on this, take a look at the vSphere Storage Guide.
I’d imagine that would be good when you have a specific vendor SATP like you have with vmw_satp_alua_cx and you want to enforce the vendor recommended PSP of RR.
NetApp for instance don’t as yet have a vendor SATP and will use the VMware default of vmw_satp_alua and I guess i’d be wary of changing the default VMware setting, even though NetApp best practise is to use the RR PSP. Their preferred method is to use the Virtual Storage Console to configure settings.
I can think of case such as MSCS where you wouldn’t want to do this but it does offer you flexibility in your own environment.
Thanks for the comment Greg. You are right about accounting for any and all arrays that are detected, especially if they happen to use the same SATP. Other claim rules can be defined which handle a heterogeneous environment. This is more about laying a foundational framework of knowledge. It’s a demonstration of a single array type and it showcases just one example of modifying the PSA components. It does go a lot deeper and of course all considerations should be on the table when tuning storage in any size environment.
This script changes it per cluster(bases on 4.1 powercli)
$vcserver = ‘vcenter’
$cluster = ‘Cluster’
$psp = “VMW_PSP_RR”
$satp = “VMW_SATP_DEFAULT_AA”
$esxCred = Get-Credential
Connect-VIServer $vcserver
$arrhosts = get-vmhost -location (get-cluster $cluster)
Foreach ($h in $arrhosts){
$hostView = get-view $h.id
$policy = new-object VMware.Vim.HostMultipathInfoLogicalUnitPolicy
$policy.policy = “rr”
$storageSystem = get-view $hostView.ConfigManager.StorageSystem
$storageSystem.StorageDeviceInfo.MultipathInfo.lun | where { $_.Path.length -gt 1 } |foreach { $storageSystem.SetMultipathLunPolicy($_.ID, $policy) }
}
Foreach ($h in $arrhosts) {
Connect-VIServer $h -Credential $esxCred | Out-Null
}
Foreach($esxcli in Get-EsxCli) {
#Change the default PSP for our SATP
$esxcli.nmp.satp.setdefaultpsp($psp,$satp) #| Out-Null
}
Foreach ($esx in Get-Cluster $cluster | Get-VMHost) {
Disconnect-VIServer $esx.name -Confirm:$false
}
Disconnect-VIServer * -Confirm:$false | Out-Null
This is very good information.Thanks
Great example, but for CLARiiON storage, you definitely do NOT want to select Round Robin. For best overall performance, you should have it set to Fixed in 5.0 (which used to be called Fixed_AP in v4.1).
@Jim Burd: That’s news to me.
For best overall performance, EMC would like you to purchase PowerPath/VE
Have a great weekend.
Well, with PowerCLI can get you the job in minutes..
Get-VMHost | Get-SCSILUNs -LunType “Disk” | where {_.MultipathPolicy -notlike “RoundRobin”} | Set-SCSILUN -MultipathPolicy RoundRobin