Intel Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA

Download PDF
ID 683633
Date 12/04/2020
Public
Document Table of Contents
Document Table of Contents x
1. About This Document 2. Introduction 3. Getting Started 4. Installing the OPAE Software Package 5. Identifying the Flash Image and BMC Firmware 6. Running FPGA Diagnostics 7. Running the OPAE in a Non-Virtualized Environment 8. Running the OPAE in a Virtualized Environment 9. Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA Archives 10. Document Revision History for Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA A. Updating the FIM and BMC Firmware B. Handling Graceful Thermal Shutdown C. FPGA Device Access Permission D. Memlock Limit E. Hugepage Settings F. Troubleshooting Frequently Asked Questions (FAQ) G. Documentation Available for the Intel® Acceleration Stack for Intel® Xeon® CPU with FPGAs 1.2.1 Release
1. About This Document x
1.1. Intended Audience 1.2. Acronym List 1.3. Acceleration Glossary
3. Getting Started x
3.1. Intel Acceleration Stack Hardware Features 3.2. System Requirements 3.3. Installing the Intel® PAC with Intel® Arria® 10 GX FPGA Card In the Host Machine 3.4. Installing the Intel Acceleration Stack
3.4. Installing the Intel Acceleration Stack x
3.4.1. Installing the Intel Acceleration Stack Runtime Package on the Host Machine 3.4.2. Installing the Intel Acceleration Stack Development Package on the Host Machine 3.4.3. Understanding the Extracted Intel® PAC with Intel® Arria® 10 GX FPGA Release Package
4. Installing the OPAE Software Package x
4.1. RHEL 7.6: Installing the OPAE Framework from Prebuilt Binaries (RPM) 4.2. Ubuntu: Installing the OPAE Framework from Prebuilt Binaries (deb) 4.3. (Optional) Building and Installing the OPAE Software from Source Code 4.4. Installing the OPAE PACSign
7. Running the OPAE in a Non-Virtualized Environment x
7.1. Loading an AFU Image into the FPGA 7.2. OPAE Sample Application Programs
7.2. OPAE Sample Application Programs x
7.2.1. Running the Hello FPGA Example
8. Running the OPAE in a Virtualized Environment x
8.1. Updating Settings Required for VFs 8.2. Configuring the VF Port on the Host 8.3. Running the Hello FPGA Example on Virtual Machine
8.3. Running the Hello FPGA Example on Virtual Machine x
8.3.1. Disconnecting the VF from the VM and Reconnecting to the PF
A. Updating the FIM and BMC Firmware x
A.1. Selecting the Correct Update Method
A.1. Selecting the Correct Update Method x
A.1.1. Updating FPGA Flash and BMC Firmware
F. Troubleshooting Frequently Asked Questions (FAQ) x
F.1. Why do I see a 'No Suitable slots found' message when running fpgaconf on my AFU image? F.2. How do I flash the FIM or program the AFU in a multicard system? F.3. Which environment variables are required? F.4. What actions do I take if I see the error message 'Error enumerating resources: no driver available'? F.5. Troubleshooting OPAE Installation on RHEL
1. About This Document
2. Introduction
3. Getting Started
4. Installing the OPAE Software Package
5. Identifying the Flash Image and BMC Firmware
6. Running FPGA Diagnostics
7. Running the OPAE in a Non-Virtualized Environment
8. Running the OPAE in a Virtualized Environment
9. Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA Archives
10. Document Revision History for Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA
A. Updating the FIM and BMC Firmware
B. Handling Graceful Thermal Shutdown
C. FPGA Device Access Permission
D. Memlock Limit
E. Hugepage Settings
F. Troubleshooting Frequently Asked Questions (FAQ)
G. Documentation Available for the Intel® Acceleration Stack for Intel® Xeon® CPU with FPGAs 1.2.1 Release

B. Handling Graceful Thermal Shutdown

Note:
  • Qualified OEM server systems provide adequate cooling for standard workloads and the use of pacd may be optional.
  • For details about using pacd, including considerations that may lead to an unexpected system reboot, refer to the pacd documentation on the OPAE web page.

The Intel® FPGA PAC Daemon (pacd) is a program that can be used to help protect the server from crashing due to hardware reaching an upper or lower non-recoverable sensor threshold. pacd is capable of monitoring any of the 23 sensors reported by Board Management Controller. pacd can be run standalone, as a daemon, or as a systemd service. When the OPAE tools extra package is installed, pacd gets placed in the OPAE binaries directory (default: /usr/bin ) along with configuration and service files – pacd.conf and pacd.service, respectively.

On startup, pacd sets its thresholds to a constant -5 degrees or 5 percent lower power under the BMC threshold values. This is to pass on the Graceful Thermal Shutdown responsibility to pacd.

pacd periodically reads the sensor values and if the values exceed the threshold, it sends a SIGHUP signal to all running processes and makes the board inaccessible from the host. The daemon waits for a configurable time specified by -c in pacd.conf, as described below, to cool down the board. After this configurable wait time elapses, the pacd service programs the specified AFU. Ensure that the AFU host application monitors for a SIGHUP signal and exits.

Note: Partial reconfiguration cannot be initiated from a Virtual Machine. Hence, pacd cannot run on a Virtual Machine.
pacd can be set up as a systemd service as follows (using a shell with elevated privileges (sudo)):
  1. Edit the pacd.conf file to update the “DefaultGBSOptions” entry with a list of AFUs appropriate for your FIM. Use the full absolute path to each AFU file and precede each file name with ‘-n’. 
    sudo vim /usr/bin/pacd.conf
    Edit entry: 
    DefaultGBSOptions=-n /home/<username>/intelrtestack/\
a10_gx_pac_ias_1_2_1/hw/samples/nlb_mode_3/bin/nlb_mode_3_unsigned.gbs
    Note:
    Optional settings include:
    • PCIe address (For example: -S, -B, -D, -F), pacd monitors all Intel® FPGA PACs matching the PCIe address components specified. For example, if you specify -B 5 only, all Intel® FPGA PACs on PCIe bus 5 becomes monitored.
    • Sensor Threshold—The thresholds are global, so specifying -T 11:95.0:93.0 monitors sensor 11 on all selected Intel® FPGA PACs. When the value exceeds 95.0, it causes the default bitstream specified with -n in pacd.conf to be programmed (PR). The sensor is considered triggereduntil its value drops below 93.0.
    • You can specify a cool down period by:
      1. Changing the CooldownInterval=<time period>
      2. Setting -c <time period> for ThresholdOptions in pacd.conf or

      If both are set, then the -c <time period> for ThresholdOptions takes precedence.

    • The Sensor Number can be found by running this command:
      sudo fpgainfo bmc

    Examine the remaining option variables and adjust as appropriate for your system.

  2. Copy pacd.conf to the default systemd service configuration directory (typically /etc/sysconfig ).
    RHEL: 
    sudo cp /usr/bin/pacd.conf /etc/sysconfig/
    Ubuntu: 
    sudo cp /usr/bin/pacd.conf /etc/default
  3. Edit the pacd.service file to update “EnvironmentFile” entry to reflect where the pacd.conf file was copied. Prepend the path name with a single dash ‘-‘, and specify the path as absolute. 
    sudo vim /usr/bin/pacd.service

    Edit entry:

    RHEL: 
    EnvironmentFile=-/etc/sysconfig/pacd.conf
    Ubuntu: 
    EnvironmentFile=-/etc/default/pacd.conf
  4. Copy pacd.service to /etc/systemd/system/pacd.service. This makes pacd visible to systemd.
    RHEL: 
    sudo cp /usr/bin/pacd.service /etc/systemd/system/
    Ubuntu: 
    sudo cp /usr/bin/pacd.service /lib/systemd/system/
  5. Start pacd as a systemd service.
    Note: Please use sudo if command cannot be run in regular user mode. 
    systemctl daemon-reload
    systemctl start pacd.service
  6. Optional: To enable pacd to re-start on boot, execute 
    systemctl enable pacd.service

    To check whether pacd has started and to check state or actions, please examine the log file (specified in pacd.conf on the “LogFile” line).

    For a full list of systemctl commands, run the following command: 
    systemctl -h
  7. To verify that the service is running, run the following command: 
    systemctl status pacd.service
  8. To stop the service: 
    systemctl stop pacd.service
Related Information
Level Two Title