Multi Channel DMA Intel® FPGA IP for PCI Express* Design Example User Guide

ID 683517
Date 7/30/2024
Public
Document Table of Contents

3.5.2.4.4.2. Create Guest VM by using QEMU

Follow the steps needed to create a guest environment and assign VF device to VM by using QEMU.

  1. Unbind the device from UIO driver:
    $./dpdk-devbind.py -u <bdf>
  2. Install vfio-pci module.
    $modprobe vfio-pci
  3. Bind the device to vfio-pci.
    1. In case the device is binded to ifc_uio, unbind with the following command:
      $ echo "<bdf>" > /sys/bus/pci/devices/\
      <bdf>/driver/unbind
      For example:
      echo "0000:01:00.0" > /sys/bus/pci/devices/\
      0000\:01\:00.0/driver/unbind
    2. Bind the device to vfio-pci.
      $ echo <PCI Vendor ID> <PCI Device ID> > /sys/bus/pci/drivers/vfio-pci/new_id
      For example:
      echo 1172 0000 > /sys/bus/pci/drivers/vfio-pci/new_id
  4. For CentOS VMs: Use QEMU version 3.0.0 rc0 on Intel machines. Creating a VM with 8 GB RAM is advisable.

    ./qemu-3.0.0-rc0/x86_64-softmmu/qemu-system-x86_64 -smp 8 -m 10240M -boot c -machine q35,kernel-irqchip=split -cpu host -enable-kvm -nographic -L /home/dev/QEMU/qemu-3.0.0-rc0/pc-bios -name offload1 -hda /home/dev/QEMU/vm1.qcow2 -device vfio-pci,host=01:00.4 -netdev type=tap,id=net6551,script=no,downscript=no,vhost=on,ifname=net6551 -device virtio-net-pci,netdev=net6551 -device intel-iommu,intremap=on,caching-mode=on -serial telnet::5551,server,nowait -object memory-backend-file,id=mem,size=10240M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc -monitor telnet::3331,server,nowait&

    On another terminal (vm)
    telnet localhost 5551
    ifconfig eth0 up
    user:root
    pass:root
    Bring up interface.
    $ifconfig eth0 up
    Assign the IP address to eth0.
    $ifconfig eth0 <1.1.1.11>
    
    On Host
    $ifconfig net6551 up
    $ifconfig net6551 <1.1.1.12>
    

    Copy the code by using scp.

    Example: $scp -r <directory> root@<GuestIP>:/ <path_to_the_directory>/

  5. For Ubuntu VMs:
    Get the qemu 7.0.0 version and compile using below steps
    wget  https://download.qemu.org/qemu-7.0.0.tar.xz 
    tar xvJf qemu-7.0.0.tar.xz 
    cd qemu-7.0.0 
     ./configure 
     make -j$nproc 
    cd .. 
    
    Get Ubuntu VM image
    wget https://cloud-images.ubuntu.com/releases/jammy/release/ubuntu-22.04-server-cloudimg-amd64.img 
    
    mv ubuntu-22.04-server-cloudimg-amd64.img ubuntu-22.04.qcow2 
    
    sudo qemu-img create -f qcow2 -F qcow2 -o  backing_file=./ubuntu-22.04.qcow2 mcdma_VM-1.qcow2 50G 
    
    qemu-img info mcdma_VM-1.qcow2 
    
    sudo genisoimage -output VM-1-cidata.iso -volid cidata -joliet -rock user-data meta-data 
    

    Command to launch the VM

    sudo qemu-7.0.0/build/x86_64-softmmu/qemu-system-x86_64 -smp 8 -machine accel=kvm,type=q35,kernel-irqchip=split -cpu host -enable-kvm -nographic -boot c -L qemu-7.0.0/build/pc-bios -cpu host -m 20G -hda mcdma_VM-1.qcow2 -cdrom VM-1-cidata.iso -device vfio-pci,host=$BDF -serial telnet::5551,server,nowait -monitor telnet::3331,server,nowait&

    Note:
    • On AMD machines, the parameter -device intel-iommu,intremap=on,caching-mode=on is not required.
    • If multiple devices are in the same group, check for ACS enablement in root port and bridge.
    • Below are the Host and Guest VM configurations used for verification:
      Table 44.  Host System Configuration
      Host System Configuration Details
      Operating System

      CentOS Linux release 7.8

      Ubuntu 22.04 LTS

      Linux Kernel

      CentOS: 3.10.0-1127.10.1.el7.28.x86_64

      Ubuntu: 5.15.0-52-generic x86_64

      CPU Cores 96
      RAM 128 GB (64 GB on single NUMA)
      Hypervisor KVM
      Qemu Version QEMU version 3.0.0-rc0
      Table 45.  Guest System Configuration
      Guest System Configuration Details
      Operating System CentOS Linux release 7.8
      Linux Kernel 3.10.0-1127.10.1.el7.28.x86_64
      CPU Cores 2
      RAM 8 GB