AXI Streaming Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 790711
Date 7/08/2024
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Features 3. Getting Started with the AXI Streaming Intel® FPGA IP for PCI Express* 4. IP Architecture and Functional Description 5. AXI Streaming Intel® FPGA IP for PCI Express* Parameters 6. Interfaces and Signals 7. Register Descriptions 8. Document Revision History for the AXI Streaming Intel® FPGA IP for PCI Express* User Guide A. Specifications B. Simulating the Design Example
1. Introduction x
1.1. Goal of the AXI Streaming Intel® FPGA IP for PCI Express* User Guide 1.2. Intended Audience for the AXI Streaming Intel® FPGA IP for PCI Express* User Guide 1.3. What is PCI Express* ? 1.4. What are the Intel® FPGA IPs for PCI Express* ? 1.5. What is the AXI Streaming Intel® FPGA IP for PCI Express* ? 1.6. Example Use Models 1.7. Design Flow Requirements
1.7. Design Flow Requirements x
1.7.1. Design Software 1.7.2. Hardware
2. Features x
2.1. Supported Features 2.2. Device Family Support 2.3. Performance
3. Getting Started with the AXI Streaming Intel® FPGA IP for PCI Express* x
3.1. Download and Install Quartus Software 3.2. Obtain and Install Intel FPGA IPs and Licenses 3.3. Configure and Generate the AXI Streaming Intel® FPGA IP for PCI Express* 3.4. About the AXI Streaming Intel® FPGA IP for PCI Express Design Examples 3.5. Instantiate and Connect the AXI Streaming Intel® FPGA IP for PCI Express* Interfaces 3.6. Simulate the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.7. Compile the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.8. Software Drivers for AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.9. Build the Application for the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.10. Verification with the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.11. Debugging with the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant
3.4. About the AXI Streaming Intel® FPGA IP for PCI Express Design Examples x
3.4.1. Functional Description for the Programmed Input/Output (PIO) Design Example 3.4.2. Functional Description for the Single Root I/O Virtualization (SR-IOV) Design Example 3.4.3. Functional Description for the Performance Design Example
3.5. Instantiate and Connect the AXI Streaming Intel® FPGA IP for PCI Express* Interfaces x
3.5.1. Clocking and Resets 3.5.2. Application Packet Datapath 3.5.3. Enabling Additional Features 3.5.4. Enabling Optional Interfaces
3.6. Simulate the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.6.1. Simulating the Design Example
3.6.1. Simulating the Design Example x
3.6.1.1. Steps to Run Simulations
3.6.1.1. Steps to Run Simulations x
3.6.1.1.1. VCS* Simulator
3.8. Software Drivers for AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.8.1. Installing the Linux Kernel Driver
3.9. Build the Application for the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.9.1. Running the Design Example
3.9.1. Running the Design Example x
3.9.1.1. Running the PIO Design Example
3.11. Debugging with the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.11.1. Debugging Link Training Issues 3.11.2. Debugging Data Transfer and Performance Issues
3.11.1. Debugging Link Training Issues x
3.11.1.1. Operating System Tools and Utilities 3.11.1.2. Signal Tap Logic Analyzer 3.11.1.3. Additional Debug Tools
3.11.2. Debugging Data Transfer and Performance Issues x
3.11.2.1. Advanced Error Reporting (AER) 3.11.2.2. Second-Level Debug Tools
4. IP Architecture and Functional Description x
4.1. Clocks and Resets 4.2. PCIe Hard IP (HIP) 4.3. HIP Interface (IF) Adaptor 4.4. Application Error Reporting 4.5. Debug Toolkit and Hard IP (HIP) Reconfiguration Interface 4.6. Configuration Space Extension 4.7. Control Shadow 4.8. Configuration Intercept Interface 4.9. Power Management 4.10. Legacy Interrupt 4.11. Credit Handling 4.12. Completion Timeout 4.13. Transaction Ordering 4.14. Page Request Service (PRS) Events 4.15. TX Non-Posted Metering Requirement on Application 4.16. MSI Pending 4.17. D-State Status 4.18. Configuration Retry Status Enable 4.19. AXI-Streaming Interface 4.20. Precision Time Measurement (PTM) [F/R-Tiles Only]
5. AXI Streaming Intel® FPGA IP for PCI Express* Parameters x
5.1. Parameter Editor Parameters
6. Interfaces and Signals x
6.1. Overview 6.2. Clocks and Resets 6.3. Application Packet Interface 6.4. Configuration Extension Bus Interface 6.5. Configuration Intercept Interface 6.6. Function Level Reset Interface 6.7. Control Shadow Interface (st_ctrlshadow) 6.8. Completion Timeout Interface (st_cplto) 6.9. Miscellaneous Signals 6.10. Control and Status Register Responder Interface (lite_csr) 6.11. Error Interface (st_err) 6.12. VF Error Flag Interface (vf_err/sent_vfnonfatalmsg) 6.13. VIRTIO PCI* Configuration Access Interface 6.14. Serial Data Signals
6.2. Clocks and Resets x
6.2.1. Interface Clock Signals 6.2.2. Interface Reset Signals
6.3. Application Packet Interface x
6.3.1. Header Format 6.3.2. Prefix Format 6.3.3. Function Number Format 6.3.4. BAR Number Format 6.3.5. Data and Header Packing Schemes 6.3.6. Application Packet Transmit Interface (st_tx) 6.3.7. Application Packet Receive Interface (st_rx) 6.3.8. Flow Control Credit Handling
6.3.5. Data and Header Packing Schemes x
6.3.5.1. HIP Native Mode Packing 6.3.5.2. Compact Packing 6.3.5.3. Simple Packing
6.3.5.1. HIP Native Mode Packing x
6.3.5.1.1. Application Logic Guidelines for the AXI Streaming TX Interface in HIP Native Mode (R-Tile) 6.3.5.1.2. Application Logic Guidelines for the AXI Streaming RX Interface in HIP Native Mode
6.3.5.2. Compact Packing x
6.3.5.2.1. Application Logic Guidelines for the AXI Streaming TX Interface in Compact Mode (P/R-Tiles) 6.3.5.2.2. Application Logic Guidelines for the AXI Streaming RX Interface in Compact Mode (P/R-Tiles)
6.3.5.3. Simple Packing x
6.3.5.3.1. Application Logic Guidelines for the AXI Streaming TX Interface in Simple Mode (P/R-Tiles) 6.3.5.3.2. Application Logic Guidelines for the AXI Streaming RX Interface in Simple Mode (P/R-Tiles)
6.3.8. Flow Control Credit Handling x
6.3.8.1. Application Transmit Flow Control Credit Interface (st_txcrdt) 6.3.8.2. Application Receive Flow Control Credit Interface (st_rxcrdt) (R-Tile Only)
6.4. Configuration Extension Bus Interface x
6.4.1. Configuration Extension Bus Request Interface (st_cebreq) 6.4.2. Configuration Extension Bus Response Interface (st_cebresp)
6.5. Configuration Intercept Interface x
6.5.1. Configuration Intercept Request Interface (st_ciireq) 6.5.2. Configuration Intercept Response Interface (st_ciiresp)
6.6. Function Level Reset Interface x
6.6.1. Function Level Reset Received Interface (st_flrrcvd) 6.6.2. Function Level Reset Completion Interface (st_flrcmpl)
6.13. VIRTIO PCI* Configuration Access Interface x
6.13.1. VIRTIO PCI* Config Access Request Interface (st_virtio_pcicfgreq) 6.13.2. VIRTIO PCI* Config Access Completion Interface (st_virtio_pcicfgcmpl)
7. Register Descriptions x
7.1. Register Address Map 7.2. PCIe Configuration Space 7.3. AXI Streaming Intel® FPGA IP for PCI Express* Soft Register Address Map
7.2. PCIe Configuration Space x
7.2.1. PCIe Configuration Space Registers
7.3. AXI Streaming Intel® FPGA IP for PCI Express* Soft Register Address Map x
7.3.1. AXI Streaming Intel® FPGA IP for PCI Express* Control Registers 7.3.2. IP Debug Registers 7.3.3. IP Performance Monitor Registers
7.3.1. AXI Streaming Intel® FPGA IP for PCI Express* Control Registers x
7.3.1.1. AXI Streaming Intel® FPGA IP for PCI Express* Version 7.3.1.2. AXI Streaming Intel® FPGA IP for PCI Express* Features 7.3.1.3. AXI Streaming Intel® FPGA IP for PCI Express* Interface Attributes 7.3.1.4. HOT PLUG GEN CTRL 7.3.1.5. POWER MANAGEMENT CTRL 7.3.1.6. LEGACY INTERRUPT CTRL 7.3.1.7. CFG REG IA CTRL 7.3.1.8. CFG REG IA FN NUM 7.3.1.9. CFG REG IA WRDATA 7.3.1.10. CFG REG IA RDDATA 7.3.1.11. PRS CTRL 7.3.1.12. MSI PENDING CTRL 7.3.1.13. MSI PENDING 7.3.1.14. D-STATE STS 7.3.1.15. CFG RETRY CTRL
7.3.2. IP Debug Registers x
7.3.2.1. HIP Status 7.3.2.2. HIP BP CYCLES 7.3.2.3. HIA BP CYCLES 7.3.2.4. APP BP CYCLES 7.3.2.5. HIA RX BP CYCLES
7.3.3. IP Performance Monitor Registers x
7.3.3.1. PERFMON CTRL 7.3.3.2. TX MRD TLP 7.3.3.3. TX MWR TLP 7.3.3.4. TX MSG TLP 7.3.3.5. TX CFGWR TLP 7.3.3.6. TX CFGRD TLP 7.3.3.7. RX MRD TLP 7.3.3.8. RX MWR TLP 7.3.3.9. RX MSG TLP 7.3.3.10. RX CFGWR TLP 7.3.3.11. RX CFGRD TLP 7.3.3.12. TX MEM DATA 7.3.3.13. TX CPL DATA 7.3.3.14. RX MEM DATA 7.3.3.15. RX CPL DATA
A. Specifications x
A.1. P-Tile Specifications A.2. F-Tile Specifications A.3. R-Tile Specifications
A.1. P-Tile Specifications x
A.1.1. P-Tile Completion Buffer Size A.1.2. Power Management A.1.3. MSI and MSI-X
A.2. F-Tile Specifications x
A.2.1. F-Tile Completion Buffer Size A.2.2. Power Management
A.3. R-Tile Specifications x
A.3.1. R-Tile Completion Buffer Size A.3.2. Power Management A.3.3. MSI and MSI-X
B. Simulating the Design Example x
B.1. Testbench
B.1. Testbench x
B.1.1. Testbench Modules B.1.2. Test Driver Module B.1.3. PIO Design Example Testbench B.1.4. Root Port BFM
1. Introduction
2. Features
3. Getting Started with the AXI Streaming Intel® FPGA IP for PCI Express*
4. IP Architecture and Functional Description
5. AXI Streaming Intel® FPGA IP for PCI Express* Parameters
6. Interfaces and Signals
7. Register Descriptions
8. Document Revision History for the AXI Streaming Intel® FPGA IP for PCI Express* User Guide
A. Specifications
B. Simulating the Design Example

3.3. Configure and Generate the AXI Streaming Intel® FPGA IP for PCI Express*

You can generate the AXI Streaming Intel® FPGA IP for PCI Express* as per the following process:

Table 6.  Generating the AXI Streaming Intel® FPGA IP for PCI Express*
AXI Streaming Intel® FPGA IP for PCI Express* in Description
Standalone mode

Refer to the steps listed below to generate the IP in standalone mode.

Design example

Refer to the steps listed below to generate the IP as part of the design example.

Intel Open FPGA Stack (OFS) reference design

You can use the Intel pre-designed and verified system level shell designs (e.g., Open FPGA Stack (OFS)) with the IP and other Intel FPGA IPs like DMA, etc., and software stack to run example workloads.

Note: For examples on connecting the various interfaces of the AXI Streaming Intel® FPGA IP for PCI Express* , you can access the following design repository: Open FPGA Stack Overview.
Note: OFS lags behind by one Quartus® Prime Pro Edition cycle, so it does not support the 24.2 version of the AXI Streaming Intel® FPGA IP for PCI Express* in the Quartus® Prime Pro Edition 24.2 release.

Following is the procedure to generate the AXI Streaming Intel® FPGA IP for PCI Express* and bring up a PCI Express* link using Quartus® Prime Pro Edition software in standalone mode.

  1. Use the Quartus® Prime Pro Edition software to create a Quartus Project and select the device. Currently, the AXI Streaming Intel® FPGA IP for PCI Express* is only supported on Agilex™ 7 devices with P-Tile (e.g., AGFB014R24B2E2V), F-Tile (e.g. AGIB027R29A1E2VR2), and R-Tile (AGIB027R29A1E2VR3).
  2. Use the following steps to specify IP core options and parameters:
    1. In the Quartus® Prime Pro Edition software, create a new project (File > New Project Wizard).
    2. Specify the Directory, Name, and Top-Level Entity.
    3. For Project Type, accept the default value, Empty project. Click Next.
    4. For Add Files click Next.
    5. For Family, Device & Board Settings under Family, select Agilex™ 7 .
    6. Select the Target Device for your design.
    7. Click Finish.
    8. In the IP Catalog, locate and add the AXI Streaming Intel® FPGA IP for PCI Express* . This IP is available under the PCI Express IPs.
    9. In the New IP Variant dialogue box, specify a name for your IP. The Parameter Editor saves the IP variation settings in a file named <your_ip>.ip
    10. Click Create. The Parameter Editor appears.
    11. Specify the parameters for your IP core variation. Refer to Parameter Editor Parameters for information about specific IP core parameters.
  3. Generate the IP:
    1. Generation dialogue box appears from the previous step. This allows you to generate the IP in the stand-alone mode.
    2. Specify output file generation options, and then click Generate. The IP variation files are generated according to your specifications.
    3. Click Close. The Parameter Editor adds the top-level.ip file to the current project automatically. If you are prompted to manually add the .ip file to the project, click Project > Add/Remove Files in Project to add the file.
  4. On the PCIe Interfaces and PCIe Interfaces Settings tabs, specify the parameters for your IP variation.
    If you want to generate the SR-IOV design example, perform the following steps to enable SR-IOV:
    1. On the PCIe Interfaces Settings tab, set the PCIe Hard IP Mode parameter to the SR-IOV-supported Tile and configuration.
    2. On the PCIe Interfaces Ports Settings tab, navigate to PCIe0 PCI Express/PCI Capabilities > PCIe0 Device > PCIe0 Multifunction and SR-IOV System Settings and:
      1. Check the Enable Multiple Physical Functions parameter.
      2. Set the Total Physical Functions (PFs) parameter to 2.
      3. Check the Enable SR-IOV Support parameter.
      4. Set the Total Virtual Functions of Physical Function 0 (PF0 VFs) parameter to 16.
      5. Set the Total Virtual Functions of Physical Function 1 (PF1 VFs) parameter to 16.
    3. On the PCIe0 Settings tab, navigate to PCIe0 Base Address Registers > PCIe0 PF0 BAR Configuration > PCIe0 PF0 BAR and:
      1. Set the BAR0 Type parameter to 64-bit prefetchable memory, 64-bit non-prefetchable memory, or 32-bit non-prefetchable memory.
      2. Set the BAR0 Size parameter to any value other than N/A.
    4. Repeat steps i and ii for PCIe0 PF0 BAR above in the PCIe0 PF0 VF BAR section.
    5. On the PCIe0 Settings tab, navigate to PCIe0 Base Address Registers > PCIe0 PF1 BAR Configuration > PCIe0 PF1 BAR and:
      1. Set the BAR0 Type parameter to 64-bit prefetchable memory, 64-bit non-prefetchable memory, or 32-bit non-prefetchable memory.
      2. Set the BAR0 Size parameter to any value other than N/A.
    6. Repeat steps i and ii for PCIe0 PF1 BAR above in the PCIe0 PF1 VF BAR section.
    7. On the PCIe0 Settings tab, navigate to PCIe0 PCI Express/PCI Capabilities > PCIe0 MSI-X > PCIe0 PF MSI-X > PCIe0 PF0 MSI-X and check the Enable MSI-X parameter.
    8. On the PCIe0 Settings tab, navigate to PCIe0 PCI Express/PCI Capabilities > PCIe0 MSI-X > PCIe0 PF MSI-X > PCIe0 PF1 MSI-X and check the Enable MSI-X parameter.
    9. On the PCIe0 Settings tab, navigate to PCIe0 Device Identification Registers and set the following parameters for the PCIe0 PF0 IDs and PCIe0 PF1 IDs:
      • Vendor ID: 0x00001172
      • Device ID: 0x00000000
      • Revision ID: 0x00000001
      • Class Code: 0x00ff0000
      • Subsystem Vendor ID: 0x00001172
      • Subsystem Device ID: 0x00000000
  5. To generate the design example, go to the Example Designs tab. Make the following selections:
    1. For Example Design Files, turn on the Simulation and Synthesis options. If you do not need these simulation or synthesis files, leaving the corresponding option(s) turned off significantly reduces the design example generation time.
    2. For General HDL Format, only Verilog is available in the current release.
    3. For Target Development Kit, select the FPGA development kit corresponding to the device being used in the project.
  6. For Currently Selected Example Design, select the PIO. SR-IOV, or Performance design example.
  7. Select Generate Example Design to create a design example that you can simulate and download to hardware. If you select one of the development boards, the device on that board overwrites the device previously selected in the Intel Quartus Prime project if the devices are different. When the prompt asks you to specify the directory for your design example, you can accept the default directory, ./<ip__example_design>, or choose another directory.
  8. Close the window when the design example generation is done. You may save your .ip file when prompted, but it is not required to be able to use the design example.
  9. Open the design example project by navigating to <project_dir>/<ip_example_design>/ and opening the file pcie_ed.qpf.
  10. Compile the design example project to generate the .sof file for the complete design example. This file is what you download to a board to perform hardware verification. For details on how to compile the design, refer to Compiling the Design Example.
  11. Close your design example project. Note that you cannot change the PCIe pin allocations in the Intel Quartus Prime project. However, to ease PCB routing, you can take advantage of the lane reversal and polarity inversion features supported by this IP.
Note:
For Design Examples in the current release of Quartus® Prime Pro Edition:
  • Only simulation support is available.
  • Only Gen5 x16 (R-Tile) and Gen4 x16 (P-Tile) are available.
Level Two Title