Configure SR-IOV and Create a Set of Virtual Functions

This article describes how to use single root I/O virtualization (SR-IOV), which enables configuration of a single physical network port to provide virtual functions (VFs) to a set of virtual machines (VMs). Depending on the Ethernet controller, you can create 63 or more VFs per physical port. Then with network data transferred directly between port and VM, bypassing the hypervisor network stack (and switch layer), traffic on the bus and processor interrupts are significantly reduced.

SR-IOV is available on a variety of Ethernet controllers across multiple operating systems (both hypervisor and guest OS). This article covers the basic steps to create VFs using SR-IOV on Fedora* 4.0.3.

Basic Steps to Configure SR-IOV

In this section, we walk through the basic steps required to configure SR-IOV.

1. Confirm that your Ethernet controller, hypervisor, and guest OS are supported by checking the FAQ for Intel® Ethernet Server Adapters with SR-IOV.

2. Confirm iommu (or Intel VT-d) is enabled in the BIOS.
   Note: iommu enables mapping of virtual memory addresses to physical addresses.

3. In the grub.conf file, turn on iommu and set it to passthrough mode:

   intel_iommu=on
   iommu=pt

4. Be sure to update grub file, then reboot. The grub file gets updated automatically if using you’re using a desktop system, however If you’re using a server system, you must enter the update-grub command before rebooting system.
5. Verify the setup by typing the steps below: 

   mesg | grep Virtualization   ##shows if VT-d is enabled
   cat /proc/cmdline    ##checks that iommu parameters were passed
   cd /root/DPDK/dpdk-16.07    ##checks for DPDK and moves to next step
   tools/dpdk-devbind.py --st    ##checks network devices status

6. Load the driver 

   modprobe uio
   insmod igb_uio.ko

7. Set up the virtual functions (VFs) - in this case on device 04, first 2 ports: 

   echo 1 > /sys/bus/pci/devices/0000\:04\:00.0/SR-IOV_numvfs
   echo 1 > /sys/bus/pci/devices/0000\:04\:00.1/SR-IOV_numvfs

8. Confirm the VF configuration 

   lspci | grep Eth  OR   ./dpdk-devbind.py --st 
   cat /sys/bus/pci/devices/0000\:04\:00.0/SR-IOV_numvfs
   cat /sys/bus/pci/devices/0000\:04\:00.1/SR-IOV_numvfs

Note: In these steps we created one VF for each port of a physical NIC. You can create many VFs depending upon your requirements and the capacity of your physical NIC. For example, if you're using the Intel 82599 10 Gigabit Ethernet NIC, you can enable maximum of 63 Virtual Functions per port. Find your NIC specification at http:/ark.intel.com. Look at the product brief for information about number of supported VFs.

The video below is a companion to this article.

Summary

This article provided a definition of SR-IOV, and provided the steps required to configure a system to share one physical NIC among multiple VMs.

SR-IOV Resources

SR-IOV Mode Utilization in a DPDK Environment

SR-IOV Technology Primer

FAQ for Intel® Ethernet Server Adapters with SR-IOV

SR-IOV and OVS-DPDK Hands-on Labs

Single-Root Input/Output Virtualization (SR-IOV) with Linux* Containers

About the Author

Nancy Yadav is a Software Dataplane (Platform) Application Engineer at Intel. She has worked on software development, algorithm optimization, and now helping customers to optimize network functions for SDN & NFV.