Intel Acceleration Stack Quick Start Guide: Intel FPGA Programmable Acceleration Card D5005

Download PDF
ID 683394
Date 7/09/2021
Public
Document Table of Contents
Document Table of Contents x
1. About this Document 2. System Requirements and Release Installation 3. Installing the OPAE Software Package 4. Identify the FPGA Interface Manager (FIM) and BMC Firmware Version 5. Running FPGA Diagnostics 6. Running the OPAE in a Non-Virtualized Environment 7. Running the OPAE in a Virtualized Environment 8. Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005 Archives 9. Document Revision History for Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005 A. Handling Graceful Thermal Shutdown B. FPGA Device Access Permission C. Memlock Limit D. Troubleshooting Frequently Asked Questions (FAQ)
2. System Requirements and Release Installation x
2.1. Unpacking the Intel® FPGA PAC 2.2. Precautions for Hardware Installation 2.3. Hardware Installation 2.4. Setting Up the Host Machine Software
2.4. Setting Up the Host Machine Software x
2.4.1. Installing Red Hat* Enterprise Linux* version 7.6 2.4.2. Installing the Intel Acceleration Stack 2.4.3. Verifying the Installation
2.4.2. Installing the Intel Acceleration Stack x
2.4.2.1. Installing the Intel Acceleration Stack Runtime Package on the Host Machine 2.4.2.2. Installing the Intel Acceleration Stack Development Package on the Host Machine
3. Installing the OPAE Software Package x
3.1. Unpacking and Installing the Intel FPGA Driver and the OPAE Software Development Kit (SDK) 3.2. Installing the PACSign
4. Identify the FPGA Interface Manager (FIM) and BMC Firmware Version x
4.1. Updating the FIM and BMC
4.1. Updating the FIM and BMC x
4.1.1. Updating the FIM and BMC using the fpgaotsu
6. Running the OPAE in a Non-Virtualized Environment x
6.1. Loading an AFU Image into the FPGA 6.2. Running the Hello FPGA Example
7. Running the OPAE in a Virtualized Environment x
7.1. Updating Settings Required for VFs 7.2. Configuring the VF Port on the Host 7.3. Running the Hello FPGA Example on Virtual Machine 7.4. Disconnecting the VF from the VM and Reconnecting to the PF
D. Troubleshooting Frequently Asked Questions (FAQ) x
D.1. Why do I see a 'No Suitable slots found' message when running fpgasupdate on my AFU image? D.2. Which environment variables are required? D.3. What actions do I take if I see the error message 'Error enumerating resources: no driver available'? D.4. Command lsmod | grep fpga shows no output after installing the OPAE driver. How to successfully install the OPAE driver? D.5. Command rpm -qa | grep opae does not return the installed opae rpm package. How to successfully install the packages? D.6. What action do I take if the Intel® FPGA PAC D5005 does not show up on the PCIe bus? D.7. Why does the PCIe not detect the Intel® FPGA PAC D5005 card?
1. About this Document
2. System Requirements and Release Installation
3. Installing the OPAE Software Package
4. Identify the FPGA Interface Manager (FIM) and BMC Firmware Version
5. Running FPGA Diagnostics
6. Running the OPAE in a Non-Virtualized Environment
7. Running the OPAE in a Virtualized Environment
8. Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005 Archives
9. Document Revision History for Intel® Acceleration Stack Quick Start Guide: Intel® FPGA PAC D5005
A. Handling Graceful Thermal Shutdown
B. FPGA Device Access Permission
C. Memlock Limit
D. Troubleshooting Frequently Asked Questions (FAQ)

A. Handling Graceful Thermal Shutdown

Note:
  • Qualified OEM server systems provide adequate cooling for standard workloads and the use of pacd may be optional.
  • For details about using pacd, including considerations that may lead to an unexpected system reboot, refer to the pacd documentation on the OPAE web page.

The Intel® FPGA PAC Daemon (pacd) is a program that can be used to help protect the server from crashing due to hardware reaching an upper or lower non-recoverable sensor threshold. pacd is capable of monitoring any of the 46 sensors reported by Board Management Controller. pacd can be run standalone, as a daemon, or as a systemd service. When the OPAE tools extra package is installed, pacd gets placed in the OPAE binaries directory (default: /usr/bin ) along with configuration and service files – pacd.conf and pacd.service, respectively.

On startup, pacd sets its thresholds to the BMC's sensor threshold values which are readjusted such that the threshold range is expanded up or down by 5%. This is to pass on the Graceful Thermal Shutdown responsibility to pacd.

pacd periodically reads the sensor values and if the values exceed the threshold, it sends a SIGHUP signal to all running processes and makes the board inaccessible from the host. The daemon waits for a configurable time specified by -c in pacd.conf, as described below, to cool down the board. After this configurable wait time elapses, the pacd service programs the specified AFU. Ensure that the AFU host application monitors for a SIGHUP signal and exits.

Note: Partial reconfiguration cannot be initiated from a Virtual Machine. Hence, pacd cannot run on a Virtual Machine.
pacd can be set up as a systemd service as follows (using a shell with elevated privileges (sudo)):
  1. Edit the pacd.conf file to update the “DefaultGBSOptions” entry with a list of AFUs appropriate for your FIM. Use the full absolute path to each AFU file and precede each file name with ‘-n’. 
    sudo vim /usr/bin/pacd.conf
    Edit entry: 
    DefaultGBSOptions=-n /home/<username>/intelrtestack/\
a10_gx_pac_ias_1_2_pv/hw/samples/nlb_mode_3/bin/nlb_mode_3_unsigned.gbs
    Edit entry: 
    DefaultGBSOptions=-n /home/<username>/intelrtestack/\
d5005_pac_ias_2_0_1*/hw/samples/nlb_mode_3/bin/nlb_mode_3_unsigned.gbs
    Note:
    Optional settings include:
    • PCIe address (For example: -S, -B, -D, -F), pacd monitors all Intel® FPGA PACs matching the PCIe address components specified. For example, if you specify -B 5 only, all Intel® FPGA PACs on PCIe bus 5 becomes monitored.
    • Sensor Threshold—The thresholds are global, so specifying -T 11:95.0:93.0 monitors sensor 11 on all selected Intel® FPGA PACs. When the value exceeds 95.0, it causes the default bitstream specified with -n in pacd.conf to be programmed (PR). The sensor is considered triggereduntil its value drops below 93.0.
    • You can specify a cool down period by:
      1. Changing the CooldownInterval=<time period>
      2. Setting -c <time period> for ThresholdOptions in pacd.conf or

      If both are set, then the -c <time period> for ThresholdOptions takes precedence.

    • The Sensor Number can be found by running this command:
      sudo fpgainfo bmc

    Examine the remaining option variables and adjust as appropriate for your system.

  2. Copy pacd.conf to the default systemd service configuration directory (typically /etc/sysconfig ). 
    sudo cp /usr/bin/pacd.conf /etc/sysconfig/
  3. Edit the pacd.service file to update “EnvironmentFile” entry to reflect where the pacd.conf file was copied. Prepend the path name with a single dash ‘-‘, and specify the path as absolute. 
    sudo vim /usr/bin/pacd.service
    Edit entry: 
    EnvironmentFile=-/etc/sysconfig/pacd.conf
  4. Copy pacd.service to /etc/systemd/system/pacd.service. This makes pacd visible to systemd. 
    sudo cp /usr/bin/pacd.service /etc/systemd/system/
  5. Start pacd as a systemd service.
    Note: Please use sudo if command cannot be run in regular user mode. 
    systemctl daemon-reload
    systemctl start pacd.service
  6. Optional: To enable pacd to re-start on boot, execute 
    systemctl enable pacd.service

    To check whether pacd has started and to check state or actions, please examine the log file (specified in pacd.conf on the “LogFile” line).

    For a full list of systemctl commands, run the following command: 
    systemctl -h
  7. To verify that the service is running, run the following command: 
    systemctl status pacd.service

    Sample Output:

  8. To stop the service: 
    systemctl stop pacd.service
Level Two Title