Intel Acceleration Stack Quick Start Guide: Intel FPGA Programmable Acceleration Card D5005

Download PDF
ID 683394
Date 7/09/2021
Public
Document Table of Contents
Document Table of Contents x
1. About this Document 2. System Requirements and Release Installation 3. Installing the OPAE Software Package 4. Identify the FPGA Interface Manager (FIM) and BMC Firmware Version 5. Running FPGA Diagnostics 6. Running the OPAE in a Non-Virtualized Environment 7. Running the OPAE in a Virtualized Environment 8. Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005 Archives 9. Document Revision History for Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005 A. Handling Graceful Thermal Shutdown B. FPGA Device Access Permission C. Memlock Limit D. Troubleshooting Frequently Asked Questions (FAQ)
2. System Requirements and Release Installation x
2.1. Unpacking the Intel® FPGA PAC 2.2. Precautions for Hardware Installation 2.3. Hardware Installation 2.4. Setting Up the Host Machine Software
2.4. Setting Up the Host Machine Software x
2.4.1. Installing Red Hat* Enterprise Linux* version 7.6 2.4.2. Installing the Intel Acceleration Stack 2.4.3. Verifying the Installation
2.4.2. Installing the Intel Acceleration Stack x
2.4.2.1. Installing the Intel Acceleration Stack Runtime Package on the Host Machine 2.4.2.2. Installing the Intel Acceleration Stack Development Package on the Host Machine
3. Installing the OPAE Software Package x
3.1. Unpacking and Installing the Intel FPGA Driver and the OPAE Software Development Kit (SDK) 3.2. Installing the PACSign
4. Identify the FPGA Interface Manager (FIM) and BMC Firmware Version x
4.1. Updating the FIM and BMC
4.1. Updating the FIM and BMC x
4.1.1. Updating the FIM and BMC using the fpgaotsu
6. Running the OPAE in a Non-Virtualized Environment x
6.1. Loading an AFU Image into the FPGA 6.2. Running the Hello FPGA Example
7. Running the OPAE in a Virtualized Environment x
7.1. Updating Settings Required for VFs 7.2. Configuring the VF Port on the Host 7.3. Running the Hello FPGA Example on Virtual Machine 7.4. Disconnecting the VF from the VM and Reconnecting to the PF
D. Troubleshooting Frequently Asked Questions (FAQ) x
D.1. Why do I see a 'No Suitable slots found' message when running fpgasupdate on my AFU image? D.2. Which environment variables are required? D.3. What actions do I take if I see the error message 'Error enumerating resources: no driver available'? D.4. Command lsmod | grep fpga shows no output after installing the OPAE driver. How to successfully install the OPAE driver? D.5. Command rpm -qa | grep opae does not return the installed opae rpm package. How to successfully install the packages? D.6. What action do I take if the Intel® FPGA PAC D5005 does not show up on the PCIe bus? D.7. Why does the PCIe not detect the Intel® FPGA PAC D5005 card?
1. About this Document
2. System Requirements and Release Installation
3. Installing the OPAE Software Package
4. Identify the FPGA Interface Manager (FIM) and BMC Firmware Version
5. Running FPGA Diagnostics
6. Running the OPAE in a Non-Virtualized Environment
7. Running the OPAE in a Virtualized Environment
8. Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005 Archives
9. Document Revision History for Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005
A. Handling Graceful Thermal Shutdown
B. FPGA Device Access Permission
C. Memlock Limit
D. Troubleshooting Frequently Asked Questions (FAQ)

7.2. Configuring the VF Port on the Host

By default, the PF controls the AFU port. The following procedure transfers AFU control to the VF. After the transfer to VF control, applications running on the VM can access the AFU.

In a multi-card system, if you want to configure the VF on only a single PCIe device, run below command to find the device mapping for the specific PCIe: 
ls -l /sys/class/fpga/intel-fpga-dev.*
Sample output: 
/sys/class/fpga/intel-fpga-dev.0 -> ../../devices/pci0000:36/0000:36:00.0/0000:37:00.0/fpga/intel-fpga-dev.0
/sys/class/fpga/intel-fpga-dev.1 -> ../../devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/fpga/intel-fpga-dev.1

To target PCIe B:D.F (AF:00.0) and B:D.F (37:00.0) in the following commands, use instance id 1 and 0 instead of * respectively.

  1. Run the following commands to export the required paths: 
    export port_path=$(find /sys/class/fpga/intel-fpga-dev.* \
 -maxdepth 1 -follow -iname intel-fpga-port.*)
export link_path=$(readlink -m /$port_path/../)
export pci_path=$link_path/../../
  2. Release the port controlled by the PF using the fpgaport tool: 
    sudo fpgaport release /dev/intel-fpga-fme.* 0
  3. Enable SR-IOV and VFs. Each VF has 1 AFU Port: 
    sudo sh -c "echo 1 > $pci_path/sriov_numvfs"
  4. Find the additional Device number for the VF Device: 
    lspci -nn | grep :0b2[bc]
    Sample output: 
    04:00.0 Processing accelerators [1200]: Intel Corporation Device [8086:0b2b]
04:00.1 Processing accelerators [1200]: Intel Corporation Device [8086:0b2c]

    lspci shows an additional Device number, 0b2c. This is the VF Device you assign to a VM. The original Bus and Device numbers for the PF remain 0b2b.

    Note that the Domain:Bus:Device.Function (BDF) notation for the VF device is: 000:04:00.1

  5. Load the vfio-pci driver: 
    sudo modprobe vfio-pci
  6. Unbind the VF Device from its driver: 
    sudo sh -c "echo 0000:04:00.1 > \
/sys/bus/pci/devices/0000:04:00.1/driver/unbind"
  7. Find the Vendor ID and Device ID for the VF Device: 
    lspci -n -s 04:00.1

    Expected output: 

    04:00.1 1200: 8086:0b2c
  8. Bind the VF to the vfio-pci driver: Use the Vendor ID and Device ID from previous step. 
    sudo sh -c "echo 8086 0b2c > \
/sys/bus/pci/drivers/vfio-pci/new_id"
Level Two Title