Should I enable jumbo frames with iSCSI?
The general recommendation is to use the standard MTU of 1500 for iSCSI connectivity.
This recommendation is predicated upon several things:
- Simplicity. Enabling jumbo frames requires setting the proper MTU throughout the entire network. This means the vSphere Switch, vmkernel port (vmknic), physical NIC (pNIC), physical switches, routers (if routed iSCSI), and finally the FlashArray target ports. It is an all too common tale to see one or more of these components missed and thus problems with stability or performance are reported.
- Not all environments benefit from jumbo frames. This was at one time a common (and rather heated) discussion in previous years. The anthem was almost always „jumbo frames enabled for best performance“. The reality though is actually based upon the workload between the initiators and target. If your applications / environment are consistently sending larger I/O requests than there is a good chance jumbo frames could help. How much will it help? Well, that answer can vary greatly so we won’t go into that here. The caveat though is that if the opposite is true (mostly smaller I/O requests), it can actually result in a performance penalty in your environment. If your host is waiting around to fill up a jumbo frame with smaller I/O requests then you are actually delaying transmission of your I/O and thus a slight performance penalty can be noted. How much? Again, it varies and isn’t the scope of this document.
The key takeaway here is know your environment. If you find jumbo frames are optimal for your environment please have all proper parties involved from end-to-end to ensure everything is implemented correctly.
If you decide to implement jumbo frames, the following command is vital to ensure you have properly configured your environment end-to-end:
vmkping -I <iscsi_vmk_interface> -d -s 8972 <ip_addr_of_target>
This ensures packets are not fragmented during the ping test (-d) and tests jumbo frames (-s 8972).