Arria® 10 and Cyclone® 10 GX Avalon® Streaming Interface for PCI Express* User Guide

Download PDF
ID 683647
Date 9/11/2024
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Quick Start Guide 3. Arria® 10 or Cyclone® 10 GX Parameter Settings 4. Physical Layout 5. Interfaces and Signal Descriptions 6. Registers 7. Reset and Clocks 8. Interrupts 9. Error Handling 10. PCI Express Protocol Stack 11. Transaction Layer Protocol (TLP) Details 12. Throughput Optimization 13. Design Implementation 14. Additional Features 15. Hard IP Reconfiguration 16. Testbench and Design Example 17. Debugging A. Transaction Layer Packet (TLP) Header Formats B. Lane Initialization and Reversal C. Arria® 10 or Cyclone® 10 GX Avalon-ST Interface for PCIe Solutions User Guide Archive D. Document Revision History
1. Datasheet x
1.1. Arria® 10 or Cyclone® 10 GX Avalon-ST Interface for PCI Express* Datasheet 1.2. Release Information 1.3. Device Family Support 1.4. Configurations 1.5. Debug Features 1.6. IP Core Verification 1.7. Resource Utilization 1.8. Recommended Speed Grades 1.9. Creating a Design for PCI Express
1.1. Arria® 10 or Cyclone® 10 GX Avalon-ST Interface for PCI Express* Datasheet x
1.1.1. Arria® 10 or Cyclone® 10 GX Features
1.6. IP Core Verification x
1.6.1. Compatibility Testing Environment
2. Quick Start Guide x
2.1. Directory Structure 2.2. Design Components 2.3. Generating the Design 2.4. Simulating the Design 2.5. Compiling and Testing the Design in Hardware
3. Arria® 10 or Cyclone® 10 GX Parameter Settings x
3.1. Parameters 3.2. Arria® 10 or Cyclone® 10 GX Avalon-ST Settings 3.3. Base Address Register (BAR) and Expansion ROM Settings 3.4. Base and Limit Registers for Root Ports 3.5. Device Identification Registers 3.6. PCI Express and PCI Capabilities Parameters 3.7. Vendor Specific Extended Capability (VSEC) 3.8. Configuration, Debug, and Extension Options 3.9. PHY Characteristics 3.10. Example Designs
3.6. PCI Express and PCI Capabilities Parameters x
3.6.1. PCI Express and PCI Capabilities 3.6.2. Error Reporting 3.6.3. Link Capabilities 3.6.4. MSI and MSI-X Capabilities 3.6.5. Slot Capabilities 3.6.6. Power Management
4. Physical Layout x
4.1. Hard IP Block Placement In Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. Interfaces and Signal Descriptions x
5.1. Clock Signals 5.2. Reset, Status, and Link Training Signals 5.3. ECRC Forwarding 5.4. Error Signals 5.5. Interrupts for Endpoints 5.6. Interrupts for Root Ports 5.7. Completion Side Band Signals 5.8. Parity Signals 5.9. LMI Signals 5.10. Transaction Layer Configuration Space Signals 5.11. Hard IP Reconfiguration Interface 5.12. Power Management Signals 5.13. Physical Layer Interface Signals
5.10. Transaction Layer Configuration Space Signals x
5.10.1. Configuration Space Register Access Timing 5.10.2. Configuration Space Register Access
5.13. Physical Layer Interface Signals x
5.13.1. Serial Data Signals 5.13.2. PIPE Interface Signals 5.13.3. Test Signals 5.13.4. Arria® 10 Development Kit Conduit Interface
6. Registers x
6.1. Correspondence between Configuration Space Registers and the PCIe Specification 6.2. Type 0 Configuration Space Registers 6.3. Type 1 Configuration Space Registers 6.4. PCI Express Capability Structures 6.5. Intel-Defined VSEC Registers 6.6. Advanced Error Reporting Capability
6.5. Intel-Defined VSEC Registers x
6.5.1. CvP Registers
6.6. Advanced Error Reporting Capability x
6.6.1. Uncorrectable Internal Error Mask Register 6.6.2. Uncorrectable Internal Error Status Register 6.6.3. Correctable Internal Error Mask Register 6.6.4. Correctable Internal Error Status Register
7. Reset and Clocks x
7.1. Reset Sequence for Hard IP for PCI Express IP Core and Application Layer 7.2. Clocks
7.2. Clocks x
7.2.1. Clock Domains 7.2.2. Clock Summary
7.2.1. Clock Domains x
7.2.1.1. coreclkout_hip 7.2.1.2. pld_clk
8. Interrupts x
8.1. Interrupts for Endpoints 8.2. Interrupts for Root Ports
8.1. Interrupts for Endpoints x
8.1.1. MSI and Legacy Interrupts 8.1.2. MSI-X 8.1.3. Implementing MSI-X Interrupts 8.1.4. Legacy Interrupts
9. Error Handling x
9.1. Physical Layer Errors 9.2. Data Link Layer Errors 9.3. Transaction Layer Errors 9.4. Error Reporting and Data Poisoning 9.5. Uncorrectable and Correctable Error Status Bits
10. PCI Express Protocol Stack x
10.1. Top-Level Interfaces 10.2. Transaction Layer 10.3. Data Link Layer 10.4. Physical Layer
10.1. Top-Level Interfaces x
10.1.1. Avalon-ST Interface 10.1.2. Clocks and Reset 10.1.3. Local Management Interface (LMI Interface) 10.1.4. Hard IP Reconfiguration 10.1.5. Interrupts 10.1.6. PIPE
10.2. Transaction Layer x
10.2.1. Configuration Space
11. Transaction Layer Protocol (TLP) Details x
11.1. Supported Message Types 11.2. Transaction Layer Routing Rules 11.3. Receive Buffer Reordering
11.1. Supported Message Types x
11.1.1. INTX Messages 11.1.2. Power Management Messages 11.1.3. Error Signaling Messages 11.1.4. Locked Transaction Message 11.1.5. Slot Power Limit Message 11.1.6. Vendor-Defined Messages 11.1.7. Hot Plug Messages
11.3. Receive Buffer Reordering x
11.3.1. Using Relaxed Ordering
12. Throughput Optimization x
12.1. Throughput of Posted Writes 12.2. Throughput of Non-Posted Reads
13. Design Implementation x
13.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 13.2. Recommended Reset Sequence to Avoid Link Training Issues 13.3. SDC Timing Constraints
14. Additional Features x
14.1. Configuration over Protocol (CvP) 14.2. Autonomous Mode 14.3. ECRC
14.2. Autonomous Mode x
14.2.1. Enabling Autonomous Mode 14.2.2. Enabling CvP Initialization
14.3. ECRC x
14.3.1. ECRC on the RX Path 14.3.2. ECRC on the TX Path
16. Testbench and Design Example x
16.1. Endpoint Testbench 16.2. Test Driver Module 16.3. Root Port BFM Overview 16.4. BFM Procedures and Functions
16.1. Endpoint Testbench x
16.1.1. Endpoint Testbench for SR-IOV
16.3. Root Port BFM Overview x
16.3.1. BFM Memory Map 16.3.2. Configuration Space Bus and Device Numbering 16.3.3. Configuration of Root Port and Endpoint 16.3.4. Issuing Read and Write Transactions to the Application Layer
16.4. BFM Procedures and Functions x
16.4.1. ebfm_barwr Procedure 16.4.2. ebfm_barwr_imm Procedure 16.4.3. ebfm_barrd_wait Procedure 16.4.4. ebfm_barrd_nowt Procedure 16.4.5. ebfm_cfgwr_imm_wait Procedure 16.4.6. ebfm_cfgwr_imm_nowt Procedure 16.4.7. ebfm_cfgrd_wait Procedure 16.4.8. ebfm_cfgrd_nowt Procedure 16.4.9. BFM Configuration Procedures 16.4.10. BFM Shared Memory Access Procedures 16.4.11. BFM Log and Message Procedures 16.4.12. Verilog HDL Formatting Functions
16.4.9. BFM Configuration Procedures x
16.4.9.1. ebfm_cfg_rp_ep Procedure 16.4.9.2. ebfm_cfg_decode_bar Procedure
16.4.10. BFM Shared Memory Access Procedures x
16.4.10.1. Shared Memory Constants 16.4.10.2. shmem_write Task 16.4.10.3. shmem_read Function 16.4.10.4. shmem_display Verilog HDL Function 16.4.10.5. shmem_fill Procedure 16.4.10.6. shmem_chk_ok Function
16.4.11. BFM Log and Message Procedures x
16.4.11.1. ebfm_display Verilog HDL Function 16.4.11.2. ebfm_log_stop_sim Verilog HDL Function 16.4.11.3. ebfm_log_set_suppressed_msg_mask Task 16.4.11.4. ebfm_log_set_stop_on_msg_mask Verilog HDL Task 16.4.11.5. ebfm_log_open Verilog HDL Function 16.4.11.6. ebfm_log_close Verilog HDL Function
16.4.12. Verilog HDL Formatting Functions x
16.4.12.1. himage1 16.4.12.2. himage2 16.4.12.3. himage4 16.4.12.4. himage8 16.4.12.5. himage16 16.4.12.6. dimage1 16.4.12.7. dimage2 16.4.12.8. dimage3 16.4.12.9. dimage4 16.4.12.10. dimage5 16.4.12.11. dimage6 16.4.12.12. dimage7
17. Debugging x
17.1. Simulation Fails To Progress Beyond Polling.Active State 17.2. Hardware Bring-Up Issues 17.3. Link Training 17.4. Creating a Signal Tap Debug File to Match Your Design Hierarchy 17.5. Use Third-Party PCIe Analyzer 17.6. BIOS Enumeration Issues
17.3. Link Training x
17.3.1. Link Hangs in L0 State
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
D. Document Revision History x
D.1. Document Revision History of the Arria® 10 or Cyclone® 10 GX Avalon® Streaming (Avalon-ST) Interface for PCIe* Solutions User Guide
1. Datasheet
2. Quick Start Guide
3. Arria® 10 or Cyclone® 10 GX Parameter Settings
4. Physical Layout
5. Interfaces and Signal Descriptions
6. Registers
7. Reset and Clocks
8. Interrupts
9. Error Handling
10. PCI Express Protocol Stack
11. Transaction Layer Protocol (TLP) Details
12. Throughput Optimization
13. Design Implementation
14. Additional Features
15. Hard IP Reconfiguration
16. Testbench and Design Example
17. Debugging
A. Transaction Layer Packet (TLP) Header Formats
B. Lane Initialization and Reversal
C. Arria® 10 or Cyclone® 10 GX Avalon-ST Interface for PCIe Solutions User Guide Archive
D. Document Revision History

5.2. Reset, Status, and Link Training Signals

Refer to Reset and Clocks for more information about the reset sequence and a block diagram of the reset logic.

Table 27.  Reset Signals

Signal

Direction

Description

npor

Input

Active low reset signal. In the Intel hardware example designs, npor is the OR of pin_perst and local_rstn coming from the software Application Layer. If you do not drive a soft reset signal from the Application Layer, this signal must be derived from pin_perst. You cannot disable this signal. Resets the entire IP Core and transceiver. Asynchronous.

This signal is edge, not level sensitive; consequently, you cannot use a low value on this signal to hold custom logic in reset. For more information about the reset controller, refer to Reset.

clr_st Output This optional reset signal has the same effect as reset_status. You enable this signal by turning On the Enable Avalon-ST reset output port in the parameter editor.
reset_status

Output

Active high reset status signal. When asserted, this signal indicates that the Hard IP clock is in reset. The reset_status signal is synchronous to the pld_clk clock and is deasserted only when the npor is deasserted and the Hard IP for PCI Express is not in reset (reset_status_hip = 0). You should use reset_status to drive the reset of your application. This reset is used for the Hard IP for PCI Express IP Core with the Avalon-ST interface.

pin_perst

Input

Active low reset from the PCIe reset pin of the device. pin_perst resets the datapath and control registers. Configuration over PCI Express (CvP) requires this signal. For more information about CvP refer to Arria® 10 CvP Initialization and Partial Reconfiguration over PCI Express User Guide.

Arria® 10 devices can have up to 4 instances of the Hard IP for PCI Express IP core. Each instance has its own pin_perst signal. Cyclone® 10 GX have a single instance of the Hard IP for PCI Express IP core. You must connect the pin_perst of each Hard IP instance to the corresponding nPERST pin of the device. These pins have the following locations:

  • NPERSTL0: bottom left Hard IP and CvP blocks
  • NPERSTL1: top left Hard IP block
  • NPERSTR0: bottom right Hard IP block
  • NPERSTR1: top right Hard IP block

For example, if you are using the Hard IP instance in the bottom left corner of the device, you must connect pin_perst to NPERSL0.

For maximum use of the Arria® 10 or Cyclone® 10 GX device, Intel recommends that you use the bottom left Hard IP first. This is the only location that supports CvP over a PCIe link. If your design does not require CvP, you may select other Hard IP blocks.

Refer to the Arria 10 GX, GT, and SX Device Family Pin Connection Guidelines or Cyclone® 10 GXDevice Family Pin Connection Guidelines for more detailed information about these pins.

Figure 30. Reset and Link Training Timing Relationships

The following figure illustrates the timing relationship between npor and the LTSSM L0 state.

Note: To meet the 100 ms system configuration time, you must use the fast passive parallel configuration scheme with CvP and a 32-bit data width (FPP x32) or use the Arria® 10 or Cyclone® 10 GX Hard IP for PCI Express in autonomous mode.
Table 28.  Status and Link Training Signals

Signal

Direction

Description

serdes_pll_locked

Output

When asserted, indicates that the PLL that generates the coreclkout_hip clock signal is locked. In pipe simulation mode this signal is always asserted.

pld_core_ready

Input

When asserted, indicates that the Application Layer is ready for operation and is providing a stable clock to the pld_clk input. If the coreclkout_hip Hard IP output clock is sourcing the pld_clk Hard IP input, this input can be connected to the serdes_pll_locked output.

pld_clk_inuse

Output

When asserted, indicates that the Hard IP Transaction Layer is using the pld_clk as its clock and is ready for operation with the Application Layer. For reliable operation, hold the Application Layer in reset until pld_clk_inuse is asserted.

dlup

Output

When asserted, indicates that the Hard IP block is in the Data Link Control and Management State Machine (DLCMSM) DL_Up state.

dlup_exit

Output

This signal is asserted low for one pld_clk cycle when the IP core exits the DLCMSM DL_Up state, indicating that the Data Link Layer has lost communication with the other end of the PCIe link and left the Up state. When this pulse is asserted, the Application Layer should generate an internal reset signal that is asserted for at least 32 cycles.

ev128ns

Output

Asserted every 128 ns to create a time base aligned activity.

ev1us

Output

Asserted every 1µs to create a time base aligned activity.

hotrst_exit

Output

Hot reset exit. This signal is asserted for 1 clock cycle when the LTSSM exits the hot reset state. This signal should cause the Application Layer to be reset. This signal is active low. When this pulse is asserted, the Application Layer should generate an internal reset signal that is asserted for at least 32 cycles.

l2_exit

Output

L2 exit. This signal is active low and otherwise remains high. It is asserted for one cycle (changing value from 1 to 0 and back to 1) after the LTSSM transitions from l2.idle to detect. When this pulse is asserted, the Application Layer should generate an internal reset signal that is asserted for at least 32 cycles.

lane_act[3:0]

Output

Lane Active Mode: This signal indicates the number of lanes that configured during link training. The following encodings are defined:

  • 4’b0001: 1 lane
  • 4’b0010: 2 lanes
  • 4’b0100: 4 lanes
  • 4’b1000: 8 lanes
currentspeed[1:0]

Output

Indicates the current speed of the PCIe link. The following encodings are defined:

  • 2b’00: Undefined
  • 2b’01: Gen1
  • 2b’10: Gen2
  • 2b’11: Gen3
ltssmstate[4:0]

Output

LTSSM state: The LTSSM state machine encoding defines the following states:

  • 00000: Detect.Quiet
  • 00001: Detect.Active
  • 00010: Polling.Active
  • 00011: Polling.Compliance
  • 00100: Polling.Configuration
  • 00101: Polling.Speed
  • 00110: config.Linkwidthstart
  • 00111: Config.Linkaccept
  • 01000: Config.Lanenumaccept
  • 01001: Config.Lanenumwait
  • 01010: Config.Complete
  • 01011: Config.Idle
  • 01100: Recovery.Rcvlock
  • 01101: Recovery.Rcvconfig
  • 01110: Recovery.Idle
  • 01111: L0
  • 10000: Disable
  • 10001: Loopback.Entry
  • 10010: Loopback.Active
  • 10011: Loopback.Exit
  • 10100: Hot.Reset
  • 10101: LOs
  • 11001: L2.transmit.Wake
  • 11010: Recovery.Speed
  • 11011: Recovery.Equalization, Phase 0
  • 11100: Recovery.Equalization, Phase 1
  • 11101: Recovery.Equalization, Phase 2
  • 11110: Recovery.Equalization, Phase 3
  • 11111: Recovery.Equalization, Done
Related Information
Level Two Title