R-tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 683501
Date 12/13/2021
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Introduction 2. IP Architecture and Functional Description 3. Advanced Features 4. Interfaces 5. Parameters 6. R-tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide Archives 7. Document Revision History for the R-tile Avalon® Streaming Intel FPGA IP for PCI Express User Guide A. Configuration Space Registers B. Root Port Enumeration C. Implementation of Address Translation Services (ATS) in Endpoint Mode D. Packets Forwarded to the User Application in TLP Bypass Mode
1. Introduction x
1.1. Overview 1.2. Features 1.3. Release Information 1.4. Device Family Support 1.5. Performance and Resource Utilization 1.6. IP Core Support Levels
1.1. Overview x
1.1.1. Standards and Specifications Compliance
2. IP Architecture and Functional Description x
2.1. Overview 2.2. Bias Temperature Instability (BTI) Protection Mode 2.3. PCIe Hard IP Mode 2.4. PIPE Direct Mode
2.3. PCIe Hard IP Mode x
2.3.1. Clocking 2.3.2. Reset 2.3.3. PCI Express Protocol Stack
2.3.3. PCI Express Protocol Stack x
2.3.3.1. PMA/PCS 2.3.3.2. Data Link Layer Overview 2.3.3.3. Transaction Layer Overview
2.4. PIPE Direct Mode x
2.4.1. Clocking 2.4.2. Reset 2.4.3. PIPE Layer
3. Advanced Features x
3.1. PCIe Port Bifurcation and PHY Channel Mapping 3.2. Virtualization Support 3.3. TLP Bypass Mode
3.2. Virtualization Support x
3.2.1. SR-IOV Support 3.2.2. VirtIO Support
3.2.1. SR-IOV Support x
3.2.1.1. SR-IOV Supported Features List 3.2.1.2. Implementation 3.2.1.3. VF to PF Mapping 3.2.1.4. Function Level Reset (FLR)
3.2.1.2. Implementation x
3.2.1.2.1. VF Error Flag Interface (for x16/x8 Cores Only)
3.2.2. VirtIO Support x
3.2.2.1. VirtIO Supported Features List 3.2.2.2. Overview 3.2.2.3. Parameters 3.2.2.4. VirtIO PCI Configuration Access Interface 3.2.2.5. Registers
3.2.2.5. Registers x
3.2.2.5.1. VirtIO Common Configuration Capability Register (Address: 0x012) 3.2.2.5.2. VirtIO Common Configuration BAR Indicator Register (Address: 0x013) 3.2.2.5.3. VirtIO Common Configuration BAR Offset Register (Address: 0x014) 3.2.2.5.4. VirtIO Common Configuration Structure Length Register (Address 0x015) 3.2.2.5.5. VirtIO Notifications Capability Register (Address: 0x016) 3.2.2.5.6. VirtIO Notifications BAR Indicator Register (Address: 0x017) 3.2.2.5.7. VirtIO Notifications BAR Offset Register (Address: 0x018) 3.2.2.5.8. VirtIO Notifications Structure Length Register (Address: 0x019) 3.2.2.5.9. VirtIO Notifications Notify Off Multiplier Register (Address: 0x01A) 3.2.2.5.10. VirtIO ISR Status Capability Register (Address: 0x02F) 3.2.2.5.11. VirtIO ISR Status BAR Indicator Register (Address: 0x030) 3.2.2.5.12. VirtIO ISR Status BAR Offset Register (Address: 0x031) 3.2.2.5.13. VirtIO ISR Status Structure Length Register (Address: 0x032) 3.2.2.5.14. VirtIO Device Specific Capability Register (Address: 0x033) 3.2.2.5.15. VirtIO Device Specific BAR Indicator Register (Address: 0x034) 3.2.2.5.16. VirtIO Device Specific BAR Offset Register (Address 0x035) 3.2.2.5.17. VirtIO Device Specific Structure Length Register (Address: 0x036) 3.2.2.5.18. VirtIO PCI Configuration Access Capability Register (Address: 0x037) 3.2.2.5.19. VirtIO PCI Configuration Access BAR Indicator Register (Address: 0x038) 3.2.2.5.20. VirtIO PCI Configuration Access BAR Offset Register (Address: 0x039) 3.2.2.5.21. VirtIO PCI Configuration Access Structure Length Register (Address: 0x03A) 3.2.2.5.22. VirtIO PCI Configuration Access Data Register (Address: 0x03B)
3.3. TLP Bypass Mode x
3.3.1. Overview 3.3.2. Register Settings for the TLP Bypass Mode 3.3.3. User Avalon Memory-mapped Interface 3.3.4. Avalon® Streaming Interface
3.3.2. Register Settings for the TLP Bypass Mode x
3.3.2.1. TLPBYPASS_ERR_EN (Address 0x1314) 3.3.2.2. TLPBYPASS_ERR_STATUS (Address 0x1310)
3.3.4. Avalon® Streaming Interface x
3.3.4.1. Configuration TLP 3.3.4.2. Transmit Interface 3.3.4.3. Receive Interface 3.3.4.4. Malformed TLP 3.3.4.5. ECRC
4. Interfaces x
4.1. Overview 4.2. Clocks and Resets 4.3. Serial Data Interface 4.4. PCI Express Mode 4.5. PIPE Direct Mode
4.2. Clocks and Resets x
4.2.1. Clocks 4.2.2. Resets
4.4. PCI Express Mode x
4.4.1. Avalon® Streaming Interface 4.4.2. Precision Time Measurement (PTM) Interface 4.4.3. Interrupt Interface 4.4.4. Hard IP Reconfiguration Interface 4.4.5. Error Interface 4.4.6. Completion Timeout Interface 4.4.7. Configuration Intercept Interface 4.4.8. Power Management Interface 4.4.9. Hard IP Status Interface 4.4.10. Page Request Services (PRS) Interface 4.4.11. Function-Level Reset (FLR) Interface 4.4.12. SR-IOV VF Error Flag Interface 4.4.13. General Purpose VSEC Interface
4.4.1. Avalon® Streaming Interface x
4.4.1.1. TLP Header and Data Alignment for the Avalon® streaming RX and TX Interfaces 4.4.1.2. Credit Control 4.4.1.3. Avalon® Streaming RX Interface 4.4.1.4. Avalon® Streaming TX Interface 4.4.1.5. Tag Allocation
4.4.1.2. Credit Control x
4.4.1.2.1. Credit Interface - Data and Header 4.4.1.2.2. Credit Initialization 4.4.1.2.3. Credit Update/Release 4.4.1.2.4. RX Flow Control Interface 4.4.1.2.5. TX Flow Control Interface
4.4.1.4. Avalon® Streaming TX Interface x
4.4.1.4.1. Avalon® Streaming TX Interface tx_st_ready_o Behavior
4.4.1.5. Tag Allocation x
4.4.1.5.1. Completion Buffer Size
4.4.3. Interrupt Interface x
4.4.3.1. Legacy Interrupts 4.4.3.2. MSI 4.4.3.3. MSI-X
4.4.3.3. MSI-X x
4.4.3.3.1. Implementing MSI-X Interrupts
4.4.8. Power Management Interface x
4.4.8.1. Endpoint D3Hot Entry 4.4.8.2. Endpoint D3Hot Exit
4.5. PIPE Direct Mode x
4.5.1. Data Signals 4.5.2. Message Channel 4.5.3. Deskew Channel 4.5.4. PIPE Direct Reset Sequence 4.5.5. PIPE Direct Speed Change 4.5.6. PIPE Lane Reset Staggering 4.5.7. Unused Lanes in PIPE Direct Mode 4.5.8. PIPE Direct Mode TX/RX Datapaths
4.5.1. Data Signals x
4.5.1.1. Transmit Signals 4.5.1.2. Receive Signals
5. Parameters x
5.1. Top-Level Settings 5.2. Core Parameters
5.2. Core Parameters x
5.2.1. Avalon Parameters 5.2.2. Base Address Registers 5.2.3. PCI Express and PCI Capabilities Parameters
5.2.3. PCI Express and PCI Capabilities Parameters x
5.2.3.1. Device Capabilities 5.2.3.2. VirtIO Parameters 5.2.3.3. Link Capabilities 5.2.3.4. Legacy Interrupt Pin Register 5.2.3.5. MSI Capabilities 5.2.3.6. MSI-X Capabilities 5.2.3.7. Slot Capabilities 5.2.3.8. Latency Tolerance Reporting (LTR) 5.2.3.9. Process Address Space ID (PASID) 5.2.3.10. Device Serial Number Capability 5.2.3.11. Page Request Service (PRS) 5.2.3.12. Access Control Service (ACS) 5.2.3.13. Power Management 5.2.3.14. Vendor Specific Extended Capability (VSEC) Registers 5.2.3.15. TLP Processing Hints (TPH) 5.2.3.16. Address Translation Services (ATS) Capabilities 5.2.3.17. Precision Time Management (PTM)
C. Implementation of Address Translation Services (ATS) in Endpoint Mode x
C.1. Sending Translated/Untranslated Requests C.2. Sending a Page Request Message from the Endpoint (EP) to the Root Complex (RC) C.3. Invalidating Requests/Completions
D. Packets Forwarded to the User Application in TLP Bypass Mode x
D.1. EP TLP Bypass Mode (Upstream) D.2. RC TLP Bypass Mode (Downstream)
1. Introduction
2. IP Architecture and Functional Description
3. Advanced Features
4. Interfaces
5. Parameters
6. R-tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide Archives
7. Document Revision History for the R-tile Avalon® Streaming Intel FPGA IP for PCI Express User Guide
A. Configuration Space Registers
B. Root Port Enumeration
C. Implementation of Address Translation Services (ATS) in Endpoint Mode
D. Packets Forwarded to the User Application in TLP Bypass Mode

1.2. Features

The R-tile Avalon® streaming IP for PCI Express* supports the following features:

  • Includes a complete protocol stack including the Transaction, Data Link, and Physical Layers implemented as a Hard IP.
  • Supports Root Port (RP), Endpoint (EP) and TLP Bypass (BP) modes.
    Table 1.  Configurations Natively Supported by R-tileEndpoint = EP; Root Port = RP; TLP Bypass = BP
    Configuration Application Interface Data Width (bits) EP/RP/BP
    Gen5/Gen4/Gen3 x16 1024 EP/RP/BP
    Gen5/Gen4/Gen3 x8x8 512 EP/RP/BP
    Gen5/Gen4/Gen3 x4x4x4x4 256 RP/BP
    PIPE Direct 64 bits per Lane N/A
    Note: Gen1/Gen2 configurations are supported via link down-training.
  • Table 2.  Topologies Supported by R-tile
    Topology\ Lane# 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
    x16 Port 0 (EP/RP/BP)
    x8x8 Port 0 (EP/RP/BP) Port 1 (EP/RP/BP)
    x4x4x4x4 Port 2 (RP/BP) Port 0 (EP/RP/BP) Port 1 (EP/RP/BP) Port 3 (RP/BP)
    PIPE Direct PIPE Direct
    Note: Port 2 is not available in the 21.4 release of Intel® Quartus® Prime, but may be available in a future release.
  • The following PIPE Direct bundle modes are supported and are selectable via the PIPE Direct Mode menu in the Parameter Editor in Intel® Quartus® Prime:
    • 1x16
    • 2x8
    • 4x4
    • 8x2
    • 16x1
    • 2x4 : 1x8
    • 4x2 : 1x8
    • 8x1 : 1x8
    • 1x8 : 2x4
    • 4x2 : 2x4
    • 8x1 : 2x4
    • 1x8 : 4x2
    • 2x4 : 4x2
    • 8x1 : 4x2
    • 1x8 : 8x1
    • 2x4 : 8x1
    • 4x2 : 8x1
    Note: 1x16 means all 16 PIPE Direct channels act in a bundle mode. 16x1 means all 16 channels act as independent channels. 2x8 means the channels are in two 8-channel bundles. 2x4 : 1x8 means the channels in the Lower 8 lanes are organized as two 4-channel bundles, while the channels in the Upper 8 lanes are in an 8-channel bundle.
  • Static port bifurcation (x8x8, x4x4x4).
    Note: Port 2 is not available in x4 mode in the 21.4 release of Intel® Quartus® Prime but may be available in a future release.
  • Supports Precision Time Management (PTM) (Endpoint only).
    Note: Only Ports 0 and 1 support PTM.
  • Supports TLP Bypass mode.
    • Supports one x16, two x8, or four x4 interfaces.
    • Supports upstream/downstream TLP bypass mode.
  • Supports up to 512-byte maximum payload size (MPS).
  • Supports up to 4096-byte (4 KB) maximum read request size (MRRS).
  • Single Virtual Channel (VC).
  • Latency Tolerance Reporting (LTR).
  • Page Request Services (PRS).
  • Completion Timeout Ranges.
  • Atomic Operations (FetchAdd/Swap/CAS).
  • Extended Tag Support.
    • 10-bit Tag Support (Maximum of 768 outstanding tags (x16) / 512 outstanding tags (x8/x4) at any given time, for all functions combined).
  • Separate Refclk with Independent Spread Spectrum Clocking (SRIS).
  • Separate Refclk with no Spread Spectrum Clocking (SRNS).
  • Common Refclk architecture.
  • PCI Express* Advanced Error Reporting (PF only).
    Note: Advanced Error Reporting is always enabled in the R-tile Avalon® streaming IP for PCIe.
  • ECRC generation and checking (when the IP is not in TLP Bypass mode).
    • Application logic needs to handle ECRC generation and checking when the IP is in TLP Bypass mode.
  • Data bus parity protection.
  • Supports D0 and D3 Hot PCIe power states.
  • Lane Margining at Receiver.
  • Retimers presence detection.
Multifunction and Virtualization Features:
  • SR-IOV support (Port 0 and Port 1 only).
    Note: Ports 0 and 1 can support 8 PFs and 2K VFs. Ports 2 and 3 do not support SR-IOV, and only support 1 PF.
  • Access Control Service (ACS) capability.
    Note: For ACS, only ports 0 and 1 are supported.
  • Alternative Routing-ID Interpretation (ARI).
  • Function Level Reset (FLR).
  • TLP Processing Hint (TPH).
    Note: TPH supports the "No Steering Tag (ST)" mode only.
  • Address Translation Services (ATS).
  • Process Address Space ID (PasID).
    Note: Scalable IO and Shared Virtual Memory (SVM) may be available in a future Intel® Quartus® Prime release.
  • Configuration Intercept Interface (CII).
  • Soft VirtIO support.
    Note: Ports 0 and 1 support soft VirtIO via CII. Ports 2 and 3 do not support VirtIO.
IP Features:
  • User packet interface with separate header, data and prefix.
  • User packet interface with a split-bus architecture where the header, data and prefix busses consist of four segments each (x16 mode only). This improves the bandwidth efficiency of this interface as it can handle up to four TLPs in any given cycle.
  • Up to 768 outstanding Non-Posted requests (x16 core only).
  • Up to 512 outstanding Non-Posted requests (x8 and x4 cores).
  • Summary of outstanding Non-Posted requests supported when 8-bit tags or 10-bit tags are enabled:
    Table 3.  Outstanding Non-Posted Requests Supported
    Ports Active Cores 8-bit Tags 10-bit Tags
    0 x16 256 768 (*)
    1 x8 256 512
    2 and 3 x4 256 512
    Note: (*): Use tags 256 to 1023.
  • Completion timeout interface.
    • The PCIe Hard IP can optionally track outgoing non-posted packets to report completion timeout information to the application.
  • You cannot change the pin allocations for the R-tile Avalon® streaming IP for PCI Express* in the Intel® Quartus® Prime project. However, this IP does support lane reversal and polarity inversion on the PCB by default.
  • Supports Autonomous Hard IP mode.
    • This mode allows the PCIe Hard IP to communicate with the Host before the FPGA configuration and entry into User mode are complete.
      Note: Unless Readiness Notifications mechanisms are used, the Root Complex and/or system software must allow at least 1 second after a Conventional Reset of a device before it may determine that a device that fails to return a Successful Completion status for a valid Configuration Request is a broken device. This period is independent of how quickly Link training completes.
  • Supports CvP Init and CvP Update.
    Note: For Gen3, Gen4 and Gen5 x16 variants, Port 0 (corresponding to lanes 0 - 15) supports the CvP features. For Gen3, Gen4 and Gen5 x8 variants, only Port 0 (corresponding to lanes 0 - 7) supports the CvP features. Port 1 (corresponding to lanes 8 - 15) does not support CvP.
  • VCS*, VCS* MX, QuestaSim* FPGA Edition and QuestaSim* are the only simulators supported in the 21.4 release of Intel® Quartus® Prime. Other simulators may be supported in a future release.
  • The R-tile PHY layer does not support sending a Beacon signal.
Note: Throughout this User Guide, the term Avalon-ST may be used as an abbreviation for the Avalon® streaming interface or IP.
Level Two Title