Intel Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA

Download PDF
ID 683633
Date 12/04/2020
Public
Document Table of Contents
Document Table of Contents x
1. About This Document 2. Introduction 3. Getting Started 4. Installing the OPAE Software Package 5. Identifying the Flash Image and BMC Firmware 6. Running FPGA Diagnostics 7. Running the OPAE in a Non-Virtualized Environment 8. Running the OPAE in a Virtualized Environment 9. Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA Archives 10. Document Revision History for Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA A. Updating the FIM and BMC Firmware B. Handling Graceful Thermal Shutdown C. FPGA Device Access Permission D. Memlock Limit E. Hugepage Settings F. Troubleshooting Frequently Asked Questions (FAQ) G. Documentation Available for the Intel® Acceleration Stack for Intel® Xeon® CPU with FPGAs 1.2.1 Release
1. About This Document x
1.1. Intended Audience 1.2. Acronym List 1.3. Acceleration Glossary
3. Getting Started x
3.1. Intel Acceleration Stack Hardware Features 3.2. System Requirements 3.3. Installing the Intel® PAC with Intel® Arria® 10 GX FPGA Card In the Host Machine 3.4. Installing the Intel Acceleration Stack
3.4. Installing the Intel Acceleration Stack x
3.4.1. Installing the Intel Acceleration Stack Runtime Package on the Host Machine 3.4.2. Installing the Intel Acceleration Stack Development Package on the Host Machine 3.4.3. Understanding the Extracted Intel® PAC with Intel® Arria® 10 GX FPGA Release Package
4. Installing the OPAE Software Package x
4.1. RHEL 7.6: Installing the OPAE Framework from Prebuilt Binaries (RPM) 4.2. Ubuntu: Installing the OPAE Framework from Prebuilt Binaries (deb) 4.3. (Optional) Building and Installing the OPAE Software from Source Code 4.4. Installing the OPAE PACSign
7. Running the OPAE in a Non-Virtualized Environment x
7.1. Loading an AFU Image into the FPGA 7.2. OPAE Sample Application Programs
7.2. OPAE Sample Application Programs x
7.2.1. Running the Hello FPGA Example
8. Running the OPAE in a Virtualized Environment x
8.1. Updating Settings Required for VFs 8.2. Configuring the VF Port on the Host 8.3. Running the Hello FPGA Example on Virtual Machine
8.3. Running the Hello FPGA Example on Virtual Machine x
8.3.1. Disconnecting the VF from the VM and Reconnecting to the PF
A. Updating the FIM and BMC Firmware x
A.1. Selecting the Correct Update Method
A.1. Selecting the Correct Update Method x
A.1.1. Updating FPGA Flash and BMC Firmware
F. Troubleshooting Frequently Asked Questions (FAQ) x
F.1. Why do I see a 'No Suitable slots found' message when running fpgaconf on my AFU image? F.2. How do I flash the FIM or program the AFU in a multicard system? F.3. Which environment variables are required? F.4. What actions do I take if I see the error message 'Error enumerating resources: no driver available'? F.5. Troubleshooting OPAE Installation on RHEL
1. About This Document
2. Introduction
3. Getting Started
4. Installing the OPAE Software Package
5. Identifying the Flash Image and BMC Firmware
6. Running FPGA Diagnostics
7. Running the OPAE in a Non-Virtualized Environment
8. Running the OPAE in a Virtualized Environment
9. Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA Archives
10. Document Revision History for Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA
A. Updating the FIM and BMC Firmware
B. Handling Graceful Thermal Shutdown
C. FPGA Device Access Permission
D. Memlock Limit
E. Hugepage Settings
F. Troubleshooting Frequently Asked Questions (FAQ)
G. Documentation Available for the Intel® Acceleration Stack for Intel® Xeon® CPU with FPGAs 1.2.1 Release

8.2. Configuring the VF Port on the Host

By default, the PF controls the AFU port. The following procedure transfers AFU control to the VF. After the transfer to VF control, applications running on the VM can access the AFU.

In a multicard system, if you want to configure the VF on only a single PCIe device, run the below command to find a device mapping for the specific PCIe device: 
ls -l /sys/class/fpga/intel-fpga-dev.*
Sample output: 
/sys/class/fpga/intel-fpga-dev.0 -> ../../devices/pci0000:36/0000:36:00.0/0000:37:00.0/fpga/intel-fpga-dev.0

/sys/class/fpga/intel-fpga-dev.1 -> ../../devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/fpga/intel-fpga-dev.1

To target PCIe B:D.F (AF:00.0) and B:D.F (37:00.0) in the following commands, use instance id 1 and 0 instead of * respectively.

  1. Run the following three commands, individually, to export the required paths: 
    export port_path=$(find /sys/class/fpga/intel-fpga-dev.* \
-maxdepth 1 -follow -iname intel-fpga-port.*)
    export link_path=$(readlink -m /$port_path/../)
    export pci_path=$link_path/../../
  2. Release the port controlled by the PF using the fpgaport tool: 
    sudo fpgaport release /dev/intel-fpga-fme.* 0
  3. Enable SR-IOV and VFs. Each VF has 1 AFU Port: 
    sudo sh -c "echo 1 > $pci_path/sriov_numvfs"
  4. Find the additional device number for the VF device: 
    lspci -nn | grep :09c[45]
    Sample output: 
    04:00.0 Processing accelerators [1200]: Intel Corporation Device [8086:09c4]
04:00.1 Processing accelerators [1200]: Intel Corporation Device [8086:09c5]

    lspci shows an additional device number, 09c5. This is the VF device you assign to a VM. The original bus and device numbers for the PF remains 09c4.

    Note that the Domain:Bus:Device.Function (BDF) notation for the VF device in this example is: 000:04:00.1. Replace this BDF with the appropriate BDF for your system.

  5. Load the vfio-pci driver: 
    sudo modprobe vfio-pci
  6. Unbind the VF device from its driver: 
    sudo sh -c "echo 0000:04:00.1 > \
/sys/bus/pci/devices/0000:04:00.1/driver/unbind"
  7. Find the vendor and device ID for the VF device: 
    lspci -n -s 04:00.1

    Sample output:

    04:00.1 1200: 8086:09c5
  8. Bind the VF to the vfio-pci driver: 
    sudo sh -c "echo 8086 09c5 > \
/sys/bus/pci/drivers/vfio-pci/new_id"
Level Two Title