10Gbps Ethernet Accelerator Functional Unit (AFU) Design Example User Guide: Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA

Download PDF
ID 683422
Date 4/30/2019
Public
Document Table of Contents
Document Table of Contents x
1. About this Document 2. Overview 3. Running the Design Example Tests 4. Using the Design Example as a Platform for Further Evaluation 5. 10Gbps Ethernet AFU Design Example User Guide Archives A. Document Revision History for 10Gbps Ethernet AFU Design Example User Guide
1. About this Document x
1.1. Intended Audience 1.2. Conventions 1.3. Acronym List 1.4. Acceleration Glossary
2. Overview x
2.1. 10GbE Design Example AFU Hardware 2.2. 10GbE Design Example AFU Software
3. Running the Design Example Tests x
3.1. Setup Prerequisites 3.2. Running 10GbE Internal Loopback Test in Single Intel® PAC System 3.3. Running 10GbE External Loopback Test in Single Intel® PAC System 3.4. Running 10GbE Intel® PAC-to-PAC Test between two connected Intel® PACs
4. Using the Design Example as a Platform for Further Evaluation x
4.1. Prerequisite while Evaluating with the Intel FPGA MAC IP 4.2. Evaluation with an Alternate MAC IP
1. About this Document
2. Overview
3. Running the Design Example Tests
4. Using the Design Example as a Platform for Further Evaluation
5. 10Gbps Ethernet AFU Design Example User Guide Archives
A. Document Revision History for 10Gbps Ethernet AFU Design Example User Guide

3.4. Running 10GbE Intel® PAC-to-PAC Test between two connected Intel® PACs

In this procedure, you can install the Intel® PACs in the same system or two separate systems with the Acceleration Stack. Unless shown otherwise, you can expect the commands to return similar outputs as the internal loopback test.

Figure 2. System Setup with Intel® PACs Installed on Same Host
Figure 3. System Setup with Intel® PACs Installed on Separate Hosts

  1. Install a QSFP+ optical module in each Intel® PAC and connect the QSFP+ ports with an optical cable.
  2. Assuming the two Intel® PACs are installed in the same system, find their PCI Bus:Device:Function mappings. 
    $ lspci | grep 09c4
    Sample output:
04:00.0 Processing accelerators:Intel Corporation Device 09c4
06:00.0 Processing accelerators:Intel Corporation Device 09c4
  3. Load the AF for the 10GbE AFU example on both Intel® PACs. 
    $ cd $OPAE_PLATFORM_ROOT
$ sudo fpgaconf hw/samples/eth_e2e_e10/bin/eth_e2e_e10.gbs -b 0x04
$ sudo fpgaconf hw/samples/eth_e2e_e10/bin/eth_e2e_e10.gbs -b 0x06
  4. cd $OPAE_PLATFORM_ROOT/hw/samples/eth_e2e_e10/sw
  5. Run the following steps on your Intel® PAC:
    1. Complie the library and application using the command: 
      $ make
    2. To configure the transceiver channel into 10G mode, write 10 to the following sysfs entry: 
      $ sudo sh -c "echo 10 > /sys/class/fpga/intel-fpga-dev.<instance_id>\
/intel-fpga-fme.<instance_id>/intel-pac-hssi.<instance_id>.\
auto/hssi_mgmt/config"

      <instance_id> represents the consecutive numbering of device, fme, and hssi instances. 

      For example:
sudo sh -c "echo 10 > /sys/class/fpga/intel-fpga-dev.0\
/intel-fpga-fme.0/intel-pac-hssi.2.auto/hssi_mgmt/config"
    3. To allow non-root users to access the 10GbE AFU instance, you can provide read and write privileges to the port (/dev/intel-fpga-port.\*) where \* denotes the respective socket. For example, to provide read and write privileges on Port 0: 
      $ sudo chmod 666 /dev/intel-fpga-port.0
    4. To resolve library dependency: 
      export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH
    5. To disable the internal loopback on B:D:F - 00:0a:0b, 
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0\
 --action=loopback_disable
      You must disable loopback on all the channels that are used in the test.
    6. To trigger the DFE: 
      sudo sh -c "echo 1 > /sys/class/fpga/intel-fpga-dev.0/intel-fpga-fme.0/intel-pac-hssi.2.auto/hssi_mgmt/dfe_kickstart"
      Verify that some of the DFE tap values are non-zero. This ensures that the script run is successful. 
      cat /sys/class/fpga/intel-fpga-dev.0/intel-fpga-fme.0/intel-pac-hssi.2.auto/hssi_mgmt/dfe_kickstart
    7. To clear PHY, transmit, and receive statistics: 
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0 --action=stat_clear
      Sample output: 
      Cleared TX stats on channel 0
Cleared RX stats on channel 0
    8. To transmit 0x1000 packets: 
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0 --action=pkt_send
      Sample output: 
      Sent 0x10000 packets on channel 0
      Note: After programming the eth_e2e_e10 AFU, the initial send of packets may drop the first packet. Subsequent packet sends do not drop any packets.
      Note: In pac_hssi_e10 [-h] [-b <bus>] [-d <device>] [-f <function>] [-m Dest. MAC] -a action, the Dest (destination) MAC address is user configurable. By default, the broadcast address is used as the destination MAC address.
    9. To get PHY, transmit and receive statistics: 
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0 --action=stat
    Note: After every hot plug/unplug of the cables, you must trigger the DFE as discussed above after disabling internal loopback.
    For more details, refer to the README file located in the sw subdirectory to: 
    $OPAE_PLATFORM_ROOT/hw/samples/eth_e2e_e10/sw/README.md

    To run this example on a virtual machine:

    1. Program the eth_e2e_e10 AFU and configure the transceiver channel to 10G mode from the Host machine by referencing the previous substeps.
    2. Disable internal loopback and trigger DFE from the Host by refereccing the previous substeps.
    3. Follow the steps in the Running the OPAE in a Virtualized Environment section of the Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA to create a virtual function and attach the virtual function to a virtual machine.
    4. Run the external loopback test on the virtual machine.
Level Two Title