Intel® Acceleration Stack User Guide: Intel FPGA Programmable Acceleration Card N3000

Download PDF
ID 683040
Date 6/14/2021
Public
Document Table of Contents
Document Table of Contents x
1. About this Document 2. System Requirements 3. Hardware Installation 4. Installing the OPAE Software 5. OPAE Tools 6. Sample Test: Native Loopback 7. Installing the Intel XL710 Driver 8. Configuring Ethernet Interfaces 9. Testing Network Loopback Using Data Plane Development Kit (DPDK) 10. Graceful Shutdown 11. Single Event Upset (SEU) 12. Document Revision History for Intel Acceleration Stack User Guide: Intel® FPGA PAC N3000 A. Troubleshooting B. Upgrade your Intel® FPGA PAC N3000 with Production Version of BMC and Intel® Arria® 10 Image C. Configure the 4.19 Kernel D. fpgabist Sample Output
1. About this Document x
1.1. Acronym List
2. System Requirements x
2.1. Cooling Requirements
3. Hardware Installation x
3.1. Installing the Intel® FPGA PAC N3000
4. Installing the OPAE Software x
4.1. Install Additional Packages 4.2. Install the Release Package 4.3. Identify the Intel® MAX® 10 Version on your Intel® FPGA PAC N3000
4.2. Install the Release Package x
4.2.1. Remove Previous OPAE Packages 4.2.2. Install the Acceleration Stack for Runtime 4.2.3. Install the Acceleration Stack for Development 4.2.4. Verify the OPAE Installation 4.2.5. Install the Configuration Files
4.3. Identify the Intel® MAX® 10 Version on your Intel® FPGA PAC N3000 x
4.3.1. FPGA Factory Image Overview
5. OPAE Tools x
5.1. Using fpgasupdate 5.2. Using fpgainfo 5.3. Test PCIe and External Memories with fpgabist 5.4. Test Network Loopback using fpgadiag
7. Installing the Intel XL710 Driver x
7.1. Updating the Intel XL710 Firmware
8. Configuring Ethernet Interfaces x
8.1. Modifying the Interface Maximum Transmission Unit (MTU) Size 8.2. Setting Forward Error Correction (FEC) Mode 8.3. Ethernet Pause Flow Control 8.4. Get Link Status and Statistics
9. Testing Network Loopback Using Data Plane Development Kit (DPDK) x
9.1. Test Using an External Traffic Generator 9.2. Test Using a Packet Generator 9.3. Troubleshooting in DPDK 9.4. Revert Back from DPDK to OPAE
10. Graceful Shutdown x
10.1. Background 10.2. Using OPAE 10.3. Using DPDK
11. Single Event Upset (SEU) x
11.1. OPAE Handling of SEU 11.2. DPDK Handling of SEU
A. Troubleshooting x
A.1. If OPAE installation verification fails, how to install OPAE manually?
B. Upgrade your Intel® FPGA PAC N3000 with Production Version of BMC and Intel® Arria® 10 Image x
B.1. Upgrading from 1.1 Beta to Production Version B.2. Upgrading from 1.1 Alpha-2 or Older to Production Version
B.1. Upgrading from 1.1 Beta to Production Version x
B.1.1. Root Entry Hash Programmed B.1.2. Root Entry Hash Not Programmed
D. fpgabist Sample Output x
D.1. For 2x2x25G Configuration D.2. For 8x10G Configuration
1. About this Document
2. System Requirements
3. Hardware Installation
4. Installing the OPAE Software
5. OPAE Tools
6. Sample Test: Native Loopback
7. Installing the Intel XL710 Driver
8. Configuring Ethernet Interfaces
9. Testing Network Loopback Using Data Plane Development Kit (DPDK)
10. Graceful Shutdown
11. Single Event Upset (SEU)
12. Document Revision History for Intel Acceleration Stack User Guide: Intel® FPGA PAC N3000
A. Troubleshooting
B. Upgrade your Intel® FPGA PAC N3000 with Production Version of BMC and Intel® Arria® 10 Image
C. Configure the 4.19 Kernel
D. fpgabist Sample Output

11.2. DPDK Handling of SEU

When the SEU errors occur, an interrupt is generated in DPDK IFPGA Rawdev driver which supports FPGA management. The interrupt handler is defined in the file $RTE_SDK/drivers/raw/ifpga/ifpga_rawdev.c as: 
static void
fme_interrupt_handler(void *param)
{
        struct opae_manager *mgr = (struct opae_manager *)param;
        IFPGA_RAWDEV_PMD_INFO("%s interrupt occurred\n", __func__);
        fme_err_handle_error0(mgr);
        fme_err_handle_nonfaterror(mgr);
        fme_err_handle_catfatal_error(mgr);
}

The function fme_err_handle_catfatal_error(mgr) handles the SEU error by causing a panic through rte_panic() function call. The implementation of this interrupt handler is provided as reference code and can be customized.

When a SEU event occurs, you receive a panic message while running the DPDK application using the following command: 
$ sudo ./x86_64-native-linuxapp-gcc/app/testpmd -l 0,1,2,3,4,5,6,7 -n 4 \
--vdev 'ifpga_rawdev_cfg0,ifpga=15:00.0,port=0' -- -i --no-numa
Figure 25. Panic Message
To recover from the SEU panic which is caused by the reference SEU event interrupt handler, follow these steps:
  1. Unbind from vfio driver:
    sudo rmmod vfio-pci
  2. Rebind to OPAE driver:
    modprobe intel-fpga-pci
echo 0000:BB:DD.F > /sys/bus/pci/drivers/intel-fpga-pci/bind
  3. Re-configure the FPGA:
    1. Extract the N3000_supplemental_files.zip which is provided as part of the Acceleration Stack Installer:
      $ unzip N3000_supplemental_files.zip 
      $ cd N3000_supplemental_files/
    2. Find PCIe Root Port:
      $ chmod +x find_RP.sh
      Sample output:
      0000:ae:00.0
0000:af:00.0
0000:b0:09.0
0000:b2:00.0 -> intel-fpga-dev.0
      The first entry in the list is the PCIe Root port. The last entry is the Intel® FPGA PAC N3000.
    3. Record the RP AER value:
      $ sudo setpci -s ae:00.0 ECAP_AER+0x08.L
00210000
      $ sudo setpci -s ae:00.0 ECAP_AER+0x14.L
000031c1
      Note: Your AER values may differ from the above responses.
    4. Disable AER:
      $ sudo setpci -s <RP BDF> ECAP_AER+0x08.L=0xffffffff
      $ sudo setpci -s <RP BDF> ECAP_AER+0x14.L=0xffffffff
    5. Trigger re-configure FPGA:
      $ sudo rsu fpga b2:00.0
    6. Enable AER , use the values obtained from step 3b:
      $ sudo setpci -s <RP BDF> ECAP_AER+0x08.L=0x00210000
      $ sudo setpci -s <RP BDF> ECAP_AER+0x14.L=0x000031c1
  4. Install vfio-pci driver.
    $ sudo modprobe vfio-pci
  5. Bind all ports to the vfio-pci driver and restart the DPDK application.
Level Two Title