GTS AXI Streaming Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 813754
Date 11/04/2024
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Features 3. Getting Started with GTS AXI Streaming IP 4. IP Architecture and Functional Description 5. IP Parameters 6. Interfaces and Signals 7. Registers 8. Document Revision History for the GTS AXI Streaming Intel® FPGA IP for PCI Express* User Guide A. Troubleshooting/Debugging B. PIPE Mode Simulation C. Implementation of Address Translation Services (ATS) in Endpoint Mode
1. Introduction x
1.1. About PCI Express* 1.2. Goal of the GTS AXI Streaming Intel® FPGA IP for PCI Express* User Guide 1.3. Example Use Models
2. Features x
2.1. Supported Features 2.2. IP Support Status 2.3. Release Information 2.4. Recommended FPGA Fabric Speed Grades 2.5. Resource Utilization 2.6. IP Core and Design Example Support Levels
3. Getting Started with GTS AXI Streaming IP x
3.1. Downloading and Installing Quartus® Prime Software 3.2. Configuring and Generating the GTS AXI Streaming IP 3.3. Configuring and Generating GTS System PLL Clocks Intel® FPGA IP 3.4. Configuring and Generating GTS Reset Sequencer Intel® FPGA IP 3.5. Instantiating and Connecting GTS AXI Streaming IP Interfaces (and Other IPs) 3.6. Simulating the GTS AXI Streaming IP Variant 3.7. Compiling the GTS AXI Streaming IP Variant
4. IP Architecture and Functional Description x
4.1. Clocking 4.2. Resets 4.3. PCIe* Hard IP 4.4. Hard IP Interface (IF) Adaptor 4.5. Interrupts 4.6. Transaction Ordering 4.7. TX Non-Posted Metering Requirement on Application 4.8. AXI4-Stream Interface 4.9. Tag Allocation 4.10. Power Management 4.11. Config Retry Status Enable 4.12. Hot-Plug 4.13. Configuration Space Extension 4.14. Page Request Service (EP only) 4.15. Precision Time Measurement (PTM) 4.16. Single Root I/O Virtualization (SR-IOV) 4.17. Transaction Layer Packet (TLP) Bypass Mode 4.18. Scalable IOV
4.1. Clocking x
4.1.1. Reference Clock
4.5. Interrupts x
4.5.1. Legacy Interrupt 4.5.2. Message Signaled Interrupts (MSI) 4.5.3. Message Signal Interrupt Extended (MSI-X)
4.8. AXI4-Stream Interface x
4.8.1. Header Format 4.8.2. Prefix Format 4.8.3. Function Number Format 4.8.4. BAR Number Format 4.8.5. Data and Header Packing Scheme
4.16. Single Root I/O Virtualization (SR-IOV) x
4.16.1. SR-IOV Implementation
4.17. Transaction Layer Packet (TLP) Bypass Mode x
4.17.1. AXI-Lite Usage for TLP Bypass Mode 4.17.2. Configuration TLP 4.17.3. Transmit Interface 4.17.4. Receive Interface 4.17.5. Malformed TLP 4.17.6. ECRC
6. Interfaces and Signals x
6.1. Overview 6.2. Clocks and Resets 6.3. AXI4-Stream Interfaces 6.4. Configuration Intercept Interface 6.5. Configuration Extension Bus (CEB) Interface 6.6. Control Shadow Interface 6.7. Transmit Flow Control Credit Interface 6.8. Completion Timeout Interface 6.9. Control and Status Register Responder Interface 6.10. Function Level Reset Interface 6.11. TLP Bypass Error Reporting Interface 6.12. Error Interface 6.13. VF Error Flag Interface 6.14. VIRTIO PCI* Configuration Access Interface 6.15. Precision Time Measurement (PTM) Interface 6.16. Serial Data Signals 6.17. Miscellaneous Signals
6.2. Clocks and Resets x
6.2.1. Interface Clock Signals 6.2.2. Interface Reset Signals
6.3. AXI4-Stream Interfaces x
6.3.1. AXI4-Stream Receive (RX) Interface 6.3.2. AXI4-Stream Transmit (TX) Interface
6.4. Configuration Intercept Interface x
6.4.1. Configuration Intercept Request Interface 6.4.2. Configuration Intercept Response Interface
6.5. Configuration Extension Bus (CEB) Interface x
6.5.1. Configuration Extension Bus Request Interface 6.5.2. Configuration Extension Bus Response Interface
6.10. Function Level Reset Interface x
6.10.1. Function Level Reset Received Interface 6.10.2. Function Level Reset Completion Interface
6.14. VIRTIO PCI* Configuration Access Interface x
6.14.1. VIRTIO PCI Configuration Access Request Interface 6.14.2. VIRTIO PCI Configuration Access Completion Interface
7. Registers x
7.1. Register Address Map 7.2. PCI Express* Configuration Space 7.3. HIP Port and Status Registers 7.4. Soft Register Address Map 7.5. Indirect Register Access 7.6. Virtual Function (VF) PCI Express* Configuration Space 7.7. Intel-Defined VSEC Capability Register
7.2. PCI Express* Configuration Space x
7.2.1. PCI Express* Configuration Header Registers 7.2.2. PCI Express* Capability Structures 7.2.3. Physical Layer 16.0 GT/s Extended Capability Structures
7.3. HIP Port and Status Registers x
7.3.1. Root Port Interrupt Registers 7.3.2. PTM Registers
7.4. Soft Register Address Map x
7.4.1. Control Registers 7.4.2. Debug Registers 7.4.3. Performance Monitor Registers
7.6. Virtual Function (VF) PCI Express* Configuration Space x
7.6.1. VF PCI-Compatible Configuration Space Header Type0 7.6.2. VF PCI Express* Capability Structure 7.6.3. VF Message Signal Interrupt Extended (MSI-X) Capability Structure 7.6.4. VF Alternative Routing ID (ARI) Capability Structure 7.6.5. VF TLP Processing Hints (TPH) Capability Structure 7.6.6. VF Address Translation Services (ATS) Capability Structure 7.6.7. VF Access Control Services (ACS) Capability Structure
7.6.1. VF PCI-Compatible Configuration Space Header Type0 x
7.6.1.1. Vendor ID and Device ID Registers 7.6.1.2. Command Registers 7.6.1.3. Status Registers 7.6.1.4. Revision ID and Class Code Register 7.6.1.5. BIST, Header Type, Latency Timer, and Cache Line Size Registers 7.6.1.6. Subsystem Vendor ID and Subsystem ID Register 7.6.1.7. Capabilities Pointer
7.6.2. VF PCI Express* Capability Structure x
7.6.2.1. PCI Express* Capability List Register 7.6.2.2. PCI Express* Device Capabilities Register 7.6.2.3. PCI Express* Device Control and Status Register 7.6.2.4. Link Capabilities Register 7.6.2.5. Link Control and Status Register 7.6.2.6. PCI Express* Device Capabilities 2 Register 7.6.2.7. PCI Express* Device Control and Status 2 Register 7.6.2.8. Link Capabilities 2 Register 7.6.2.9. Link Control and Status 2 Register
7.6.3. VF Message Signal Interrupt Extended (MSI-X) Capability Structure x
7.6.3.1. MSI-X Control Register 7.6.3.2. MSI-X Table Offset Register 7.6.3.3. MSI-X Pending Bit Array Register
7.6.4. VF Alternative Routing ID (ARI) Capability Structure x
7.6.4.1. ARI Enhanced Capability Header Register 7.6.4.2. ARI Capability and Control Registers
7.6.5. VF TLP Processing Hints (TPH) Capability Structure x
7.6.5.1. TPH Requester Enhanced Capability Header 7.6.5.2. TPH Requester Capability Register 7.6.5.3. TPH Requester Control Register
7.6.6. VF Address Translation Services (ATS) Capability Structure x
7.6.6.1. ATS Enhanced Capability Header 7.6.6.2. ATS Capability Register and Control Register
7.6.7. VF Access Control Services (ACS) Capability Structure x
7.6.7.1. ACS Extended Capability Header 7.6.7.2. ACS Capability and Control Register 7.6.7.3. Egress Control Vector
A. Troubleshooting/Debugging x
A.1. Hardware A.2. Debug Toolkit
A.1. Hardware x
A.1.1. Debugging Link Training Issues A.1.2. Debugging Data Transfer and Performance Issues
A.1.1. Debugging Link Training Issues x
A.1.1.1. Generic Tools and Utilities A.1.1.2. Signal Tap Logic Analyzer
A.2. Debug Toolkit x
A.2.1. Overview A.2.2. Enabling the Agilex™ 5 Debug Toolkit A.2.3. Launching the Agilex™ 5 Debug Toolkit A.2.4. Using the Agilex™ 5 Debug Toolkit A.2.5. Using the Agilex™ 5 Link Inspector
A.2.4. Using the Agilex™ 5 Debug Toolkit x
A.2.4.1. Main View A.2.4.2. Toolkit Parameters A.2.4.3. Channel Parameters A.2.4.4. Eye Viewer
A.2.4.2. Toolkit Parameters x
A.2.4.2.1. Agilex™ 5 Information A.2.4.2.2. Event Counter A.2.4.2.3. P0 Configuration Space
A.2.4.3. Channel Parameters x
A.2.4.3.1. General PHY A.2.4.3.2. TX Path A.2.4.3.3. RX Path
C. Implementation of Address Translation Services (ATS) in Endpoint Mode x
C.1. Sending Translated/Untranslated Requests C.2. Sending a Page Request Message from the Endpoint (EP) to the Root Complex (RC) C.3. Invalidating Requests/Completions
1. Introduction
2. Features
3. Getting Started with GTS AXI Streaming IP
4. IP Architecture and Functional Description
5. IP Parameters
6. Interfaces and Signals
7. Registers
8. Document Revision History for the GTS AXI Streaming Intel® FPGA IP for PCI Express* User Guide
A. Troubleshooting/Debugging
B. PIPE Mode Simulation
C. Implementation of Address Translation Services (ATS) in Endpoint Mode

4.5.3. Message Signal Interrupt Extended (MSI-X)

The use applicatio geeates MSI-X messages which ae sigle-Dwod memoy wites. The MSI-X Capability stuctue poits to a MSI-X table stuctue ad a MSI-X Pedig Bit Aay (PBA) stuctue which ae stoed i memoy. This scheme is diffeet tha the MSI Capability stuctue, which cotais all the cotol ad status ifomatio fo the iteupts.

The GTS AXI Steamig IP povides a Cofiguatio Itecept Iteface. Use soft logic moitos this iteface to get MSI-X Eable ad MSI-X fuctio mask elated ifomatio. Use applicatio logic implemets the MSI-X tables fo all PFs ad VFs at the memoy space poited to by the BARs as a pat of you Applicatio Laye. Fo moe ifomatio o the MSI-X elated ifomatio that you ca obtai fom the Cofiguatio Itecept Iteface, efe to the MSI-X Registes sectio i the PCI Expess* Cofiguatio Space sectio.

MSI-X is a optioal featue that allows the use applicatio to suppot lage umbe of vectos with idepedet message data ad addess fo each vecto. Whe MSI-X is suppoted, you must specify the size ad the locatio (BARs ad offsets) of the MSI-X table ad PBA.

MSI-X ca suppot up to 2048 vectos pe fuctio vesus 32 vectos pe fuctio fo MSI. A fuctio is allowed to sed MSI-X messages whe MSI-X is eabled ad the fuctio is ot masked. The applicatio uses Cofiguatio Itecept Iteface to access this ifomatio.

Whe the applicatio eeds to geeate a MSI-X, it uses the cotets of the MSI-X Table (Addess ad Data) ad geeates a Memoy Wite though the GTS AXI Steamig iteface.

If you eable the MSI-X iteupt, you must implemet the MSI-X table stuctues at the memoy space poited to by the BARs as a pat of you Applicatio Laye. The MSI-X Capability Stuctue cotais ifomatio about the MSI-X Table ad PBA Stuctue. Fo example, it cotais poites to the bases of the MSI-X Table ad PBA Stuctue, expessed as offsets fom the addesses i the fuctio's BARs. The Message Cotol egiste withi the MSI-X Capability Stuctue also cotais the MSI-X Eable bit, the Fuctio Mask bit, ad the size of the MSI-X Table.

MSI-X iteupts ae stadad Memoy Wites, theefoe Memoy Wite odeig ules apply.

Table 13.  MSI-X Cofiguatio Example
MSI-X Vecto MSI-X Uppe Addess MSI-X Lowe Addess MSI-X Data
0 0x00000001 0xAAAA0000 0x00000001
1 0x00000001 0xBBBB0000 0x00000002
2 0x00000001 0xCCCC0000 0x00000003
Table 14.  PBA Table Example
PBA Table PBA Eties
Offset 0 0x0

If the applicatio eeds to geeate a MSI-X iteupt (vecto 1), it eads the MSI-X Table ifomatio, geeates a MWR TLP though the Avalo® steamig iteface ad assets the coespodig PBA bits (bit[1]) i a simila fashio as fo MSI geeatio.

The geeated TLP is set to addess 0x00000001_BBBB0000 ad the data is 0x00000002. Whe the MSI-X has bee set, the applicatio ca clea the associated PBA bits. The MSI-X capability stuctue poits to the MSI-X Table stuctue ad MSI-X Pedig Bit Aay (PBA) egistes. The BIOS sets up the statig addess offsets ad BAR associated with the poite to the statig addess of the MSI-X Table ad PBA egistes.

Figue 20. MSI-X Iteupt Compoet
  1. Host softwae sets up the MSI-X iteupts i the Applicatio Laye by completig the followig steps:
    • Host softwae eads the Message Cotol egiste at 0x050 egiste to detemie the MSI-X Table size. The umbe of table eties is the <value ead> + 1. The maximum table size is 2048 eties. Each 16-byte ety is divided i 4 fields as show i the figue below. Fo multi-fuctio vaiats, BAR4 accesses the MSI-X table. Fo all othe vaiats, ay BAR ca access the MSI-X table. The base addess of the MSI-X table must be aliged to a 4 KB bouday.
    • The host sets up the MSI-X table. It pogams MSI-X addess, data, ad masks bits fo each ety as show i the figue below.
      Figue 21. Fomat of MSI-X Table
    • The host calculates the addess of the <th> ety usig the followig fomula: 
      th_addess = base addess[BAR]+16<>
  2. Whe Applicatio Laye eeds to issue a iteupt, it dives a iteupt equest to the IRQ Souce module.
  3. 3. The IRQ Souces sets appopiate bit i the MSI-X PBA table. The PBA ca use qwod o dwod accesses. Fo qwod accesses, the IRQ Souce calculates the addess of the <mth> bit usig the followig fomulas: 
    qwod addess = <PBA base add> + 8(floo(<m>/64))
qwod bit = <m> mod 64
    Figue 22. MSI-X PBA Table
  4. The IRQ Pocesso eads the ety i the MSI-X Table.
    • If the iteupt is masked by the Vecto_Cotol field of the MSI-X table, the iteupt emais i the pedig state.
    • If the iteupt is ot masked, IRQ Pocesso seds Memoy Wite Request to the TX slave iteface. It uses the addess ad data fom the MSI-X table. If Message Uppe Addess =0, the IRQ Pocesso ceates a thee-dwod heade. If the Message Uppe Addess > 0, it ceates a 4-dwod heade.
  5. The host iteupt sevice outie detects the TLP as a iteupt ad sevice it.
Related Ifomatio
Level Two Title