Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide

Download PDF
ID 683686
Date 9/12/2024
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Getting Started with the SR-IOV Design Example 3. Parameter Settings 4. Physical Layout 5. Interfaces and Signal Descriptions 6. Registers 7. Reset and Clocks 8. Programming and Testing SR-IOV Bridge MSI Interrupts 9. Error Handling 10. IP Core Architecture 11. Design Implementation 12. Debugging 13. Document Revision History A. Transaction Layer Packet (TLP) Header Formats B. Arria® 10 Avalon-ST with SR-IOV Interface for PCIe Solutions User Guide Archive
1. Datasheet x
1.1. Arria® 10 Avalon® -ST Interface with SR-IOV for PCI Express* Datasheet 1.2. Release Information 1.3. Device Family Support 1.4. Debug Features 1.5. IP Core Verification 1.6. Performance and Resource Utilization 1.7. Recommended Speed Grades for SR-IOV Interface
1.1. Arria® 10 Avalon® -ST Interface with SR-IOV for PCI Express* Datasheet x
1.1.1. SR-IOV Features
1.5. IP Core Verification x
1.5.1. Compatibility Testing Environment
2. Getting Started with the SR-IOV Design Example x
2.1. Directory Structure for Arria® 10 SR-IOV Design Example 2.2. Design Components for the SR-IOV Design Example 2.3. Generating the SR-IOV Design Example 2.4. Compiling and Simulating the Design for SR-IOV
3. Parameter Settings x
3.1. Parameters 3.2. Arria® 10 Avalon-ST Settings 3.3. Arria® 10 SR-IOV System Settings 3.4. Base Address Register (BAR) Settings 3.5. SR-IOV Device Identification Registers 3.6. Arria® 10 Interrupt Capabilities 3.7. Physical Function TLP Processing Hints (TPH) 3.8. Address Translation Services (ATS) 3.9. PCI Express and PCI Capabilities Parameters 3.10. PHY Characteristics 3.11. Example Designs
3.9. PCI Express and PCI Capabilities Parameters x
3.9.1. PCI Express and PCI Capabilities 3.9.2. Error Reporting 3.9.3. Link Capabilities 3.9.4. Slot Capabilities 3.9.5. Power Management
4. Physical Layout x
4.1. Hard IP Block Placement In Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. Interfaces and Signal Descriptions x
5.1. Avalon-ST TX Interface 5.2. Component-Specific Avalon-ST Interface Signals 5.3. Avalon-ST RX Interface 5.4. BAR Hit Signals 5.5. Configuration Status Interface 5.6. Clock Signals 5.7. Function-Level Reset (FLR) Interface 5.8. SR-IOV Interrupt Interface 5.9. Configuration Extension Bus (CEB) Interface 5.10. Implementing MSI-X Interrupts 5.11. Control Shadow Interface 5.12. Local Management Interface (LMI) Signals 5.13. Reset, Status, and Link Training Signals 5.14. Hard IP Reconfiguration Interface 5.15. Serial Data Signals 5.16. Test Signals 5.17. PIPE Interface Signals 5.18. Arria® 10 Development Kit Conduit Interface
5.9. Configuration Extension Bus (CEB) Interface x
5.9.1. Vital Product Data (VPD) Capability
5.9.1. Vital Product Data (VPD) Capability x
5.9.1.1. Determine the Pointer Address of an External Capability Register 5.9.1.2. VPD Capability Implementation
6. Registers x
6.1. Addresses for Physical and Virtual Functions 6.2. Correspondence between Configuration Space Registers and the PCIe Specification 6.3. PCI and PCI Express Configuration Space Registers 6.4. MSI Registers 6.5. MSI-X Capability Structure 6.6. Power Management Capability Structure 6.7. PCI Express Capability Structure 6.8. Advanced Error Reporting (AER) Enhanced Capability Header Register 6.9. Uncorrectable Error Status Register 6.10. Uncorrectable Error Mask Register 6.11. Uncorrectable Error Severity Register 6.12. Correctable Error Status Register 6.13. Correctable Error Mask Register 6.14. Advanced Error Capabilities and Control Register 6.15. Header Log Registers 0-3 6.16. SR-IOV Virtualization Extended Capabilities Registers 6.17. Virtual Function Registers
6.3. PCI and PCI Express Configuration Space Registers x
6.3.1. Type 0 Configuration Space Registers 6.3.2. PCI and PCI Express Configuration Space Register Content 6.3.3. Interrupt Line and Interrupt Pin Register
6.16. SR-IOV Virtualization Extended Capabilities Registers x
6.16.1. SR-IOV Virtualization Extended Capabilities Registers Address Map 6.16.2. ARI Enhanced Capability Header 6.16.3. SR-IOV Enhanced Capability Registers 6.16.4. Initial VFs and Total VFs Registers 6.16.5. VF Device ID Register 6.16.6. Page Size Registers 6.16.7. VF Base Address Registers (BARs) 0-5 6.16.8. Secondary PCI Express Extended Capability Header 6.16.9. Lane Status Registers 6.16.10. Transaction Processing Hints (TPH) Requester Enhanced Capability Header
7. Reset and Clocks x
7.1. Reset Sequence for Hard IP for PCI Express IP Core and Application Layer 7.2. Function Level Reset (FLR) 7.3. Clocks
7.3. Clocks x
7.3.1. Clock Domains 7.3.2. Clock Summary
7.3.1. Clock Domains x
7.3.1.1. coreclkout_hip 7.3.1.2. pld_clk
8. Programming and Testing SR-IOV Bridge MSI Interrupts x
8.1. Setting Up and Verifying MSI Interrupts 8.2. Masking MSI Interrupts 8.3. Dropping a Pending MSI Interrupt
9. Error Handling x
9.1. Physical Layer Errors 9.2. Data Link Layer Errors 9.3. Transaction Layer Errors 9.4. Error Reporting and Data Poisoning 9.5. Uncorrectable and Correctable Error Status Bits
10. IP Core Architecture x
10.1. PCI Express Protocol Stack 10.2. Data Link Layer 10.3. Physical Layer 10.4. Top-Level Interfaces 10.5. Arria® 10 Hard IP for PCI Express with Single-Root I/O Virtualization (SR-IOV)
10.4. Top-Level Interfaces x
10.4.1. Avalon-ST Interface 10.4.2. Clocks and Reset 10.4.3. Interrupts 10.4.4. PIPE
11. Design Implementation x
11.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 11.2. Recommended Reset Sequence to Avoid Link Training Issues 11.3. SDC Timing Constraints
12. Debugging x
12.1. Setting Up Simulation 12.2. Simulation Fails To Progress Beyond Polling.Active State 12.3. Hardware Bring-Up Issues 12.4. Link Training 12.5. Creating a Signal Tap Debug File to Match Your Design Hierarchy 12.6. Use Third-Party PCIe Analyzer 12.7. BIOS Enumeration Issues
12.1. Setting Up Simulation x
12.1.1. Changing Between Serial and PIPE Simulation 12.1.2. Using the PIPE Interface for Gen1 and Gen2 Variants 12.1.3. Viewing the Important PIPE Interface Signals 12.1.4. Disabling the Scrambler for Gen1 and Gen2 Simulations 12.1.5. Disabling 8B/10B Encoding and Decoding for Gen1 and Gen2 Simulations
13. Document Revision History x
13.1. Document Revision History for the Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
1. Datasheet
2. Getting Started with the SR-IOV Design Example
3. Parameter Settings
4. Physical Layout
5. Interfaces and Signal Descriptions
6. Registers
7. Reset and Clocks
8. Programming and Testing SR-IOV Bridge MSI Interrupts
9. Error Handling
10. IP Core Architecture
11. Design Implementation
12. Debugging
13. Document Revision History
A. Transaction Layer Packet (TLP) Header Formats
B. Arria® 10 Avalon-ST with SR-IOV Interface for PCIe Solutions User Guide Archive

5.13. Reset, Status, and Link Training Signals

Table 35.  Reset Signals

Signal

Direction

Description

npor

Input

Active low reset signal. In the Intel hardware example designs, npor is the OR of pin_perst and local_rstn coming from the software Application Layer. If you do not drive a soft reset signal from the Application Layer, this signal must be derived from pin_perst. You cannot disable this signal. Resets the entire Arria® 10 Hard IP for PCI Express IP Core and transceiver. Asynchronous.

When CvP is enabled, an embedded hard reset controller triggers after the internal status signal indicates that the periphery image is loaded. This embedded reset does not trigger off of pin_perst.

In systems that use the hard reset controller, this signal is edge, not level sensitive; consequently, you cannot use a low value on this signal to hold custom logic in reset. For more information about the hard and soft reset controllers, refer to Reset.

pin_perst

Input

Active low reset from the PCIe reset pin of the device.

It resets the datapath and control registers. This signal is required for Configuration via Protocol (CvP). For more information about CvP refer to Configuration via Protocol (CvP).

Arria® 10 devices have up to 4 instances of the Hard IP for PCI Express. Each instance has its own pin_perst signal. Cyclone® 10 GX have a signal instance of the Hard IP for PCI Express.

Every Arria® 10 device has 4 nPERST pins, even devices with fewer than 4 instances of the Hard IP for PCI Express. You must connect the pin_perst of each Hard IP instance to the corresponding nPERST pin of the device. These pins have the following locations:

  • nPERSTL0: bottom left Hard IP and CvP blocks
  • nPERSTL1: top left Hard IP block
  • nPERSTR0: bottom right Hard IP block
  • nPERSTR1: top right Hard IP block

For example, if you are using the Hard IP instance in the bottom left corner of the device, you must connect pin_perst to nPERSL0.

For maximum use of the Arria® 10 device, Intel recommends that you use the bottom left Hard IP first. This is the only location that supports CvP over a PCIe link.

Refer to the Arria® 10 GX, GT, and SX Device Family Pin Connection Guidelines or Cyclone® 10 GX Device Family Pin Connection Guidelinesfor more detailed information about these pins.

Figure 36. Reset and Link Training Timing Relationships

The following figure illustrates the timing relationship between npor and the LTSSM L0 state.

Table 36.  Hard IP Reset Status Signals

Signal

Direction

Description

pld_clk_inuse

Output

When asserted, indicates that the Hard IP Transaction Layer is using the pld_clk as its clock and is ready for operation with the Application Layer. For reliable operation, hold the Application Layer in reset until pld_clk_inuse is asserted.

pld_core_ready

Input

When asserted, indicates that the Application Layer is ready for operation and is providing a stable clock to the pld_clk input. If the coreclkout_hip Hard IP output clock is sourcing the pld_clk Hard IP input, this input can be connected to the serdes_pll_locked output.

reset_status

Output

Active high reset status signal. When asserted, this signal indicates that the Hard IP clock is in reset. The reset_status signal is synchronous to the pld_clk clock and is deasserted only when the npor is deasserted and the Hard IP for PCI Express is not in reset (reset_status_hip = 0). You should use reset_status to drive the reset of your Application Layer. It resets the Hard IP at power-up, for hot reset and link down events.

serdes_pll_locked

Output

When asserted, indicates that the PLL that generates the coreclkout_hip clock signal is locked. In pipe simulation mode this signal is always asserted.

testin_zero

Output

When asserted, indicates accelerated initialization for simulation is active.
Table 37.  Status and Link Training SignalsThe following table describes additional signals related to the reset function for the including the ltsssm_state[4:0] bus that indicates the current link training state. These signals are not top-level signals of the Arria® 10 Hard IP for PCI Express IP Core with SR-IOV. They are listed here to assist in debugging link training issues.

Signal

Direction

Description

cfg_par_err

Output

Indicates that a parity error in a TLP routed to the internal Configuration Space. You must reset the Hard IP if this error occurs.

derr_cor_ext_rcv

Output

Indicates a corrected error in the RX buffer. This signal is for debug only. It is not valid until the RX buffer is filled with data. This is a pulse, not a level, signal. Internally, the pulse is generated with the 500 MHz clock. A pulse extender extends the signal so that the FPGA fabric running at 250 MHz can capture it. Because the error was corrected by the IP core, no Application Layer intervention is required. 4

derr_cor_ext_rpl

Output

Indicates a corrected ECC error in the retry buffer. This signal is for debug only. Because the error was corrected by the IP core, no Application Layer intervention is required. 4 (4)

derr_rpl

Output

Indicates an uncorrectable error in the retry buffer. This signal is for debug only. (4)

dlup

Output

When asserted, indicates that the Hard IP block is in the Data Link Control and Management State Machine (DLCMSM) DL_Up state.

dlup_exit

Output

This signal is asserted low for one pld_clk cycle when the IP core exits the DLCMSM DL_Up state, indicating that the Data Link Layer has lost communication with the other end of the PCIe link and left the Up state. When this pulse is asserted, the Application Layer should generate an internal reset signal that is asserted for at least 32 cycles.

ev128ns

Output

Asserted every 128 ns to create a time base aligned activity.

ev1us

Output

Asserted every 1 µs to create a time base aligned activity.

hotrst_exit

Output

Hot reset exit. This signal is asserted for 1 clock cycle when the LTSSM exits the hot reset state. This signal should cause the Application Layer to be reset. This signal is active low. When this pulse is asserted, the Application Layer should generate an internal reset signal that is asserted for at least 32 cycles.

int_status[3:0]

Output

These signals drive legacy interrupts to the Application Layer as follows:

  • int_status[0]: interrupt signal A
  • int_status[1]: interrupt signal B
  • int_status[2]: interrupt signal C
  • int_status[3]: interrupt signal D
l2_exit

Output

L2 exit. This signal is active low and otherwise remains high. It is asserted for one cycle (changing value from 1 to 0 and back to 1) after the LTSSM transitions from l2.idle to detect. When this pulse is asserted, the Application Layer should generate an internal reset signal that is asserted for at least 32 cycles.

lane_act[3:0]

Output

Lane Active Mode: This signal indicates the number of lanes that configured during link training. The following encodings are defined:

  • 4’b0001: 1 lane
  • 4’b0010: 2 lanes
  • 4’b0100: 4 lanes
  • 4’b1000: 8 lanes
ltssmstate[4:0]

Output

LTSSM state: The LTSSM state machine encoding defines the following states:

  • 00000: Detect.Quiet
  • 00001: Detect.Active
  • 00010: Polling.Active
  • 00011: Polling.Compliance
  • 00100: Polling.Configuration
  • 00101: Polling.Speed
  • 00110: config.Linkwidthstart
  • 00111: Config.Linkaccept
  • 01000: Config.Lanenumaccept
  • 01001: Config.Lanenumwait
  • 01010: Config.Complete
  • 01011: Config.Idle
  • 01100: Recovery.Rcvlock
  • 01101: Recovery.Rcvconfig
  • 01110: Recovery.Idle
  • 01111: L0
  • 10000: Disable
  • 10001: Loopback.Entry
  • 10010: Loopback.Active
  • 10011: Loopback.Exit
  • 10100: Hot.Reset
  • 10101: L0s
  • 11001: L2.transmit.Wake
  • 11010: Recovery.Speed
  • 11011: Recovery.Equalization, Phase 0
  • 11100: Recovery.Equalization, Phase 1
  • 11101: Recovery.Equalization, Phase 2
  • 11110: Recovery.Equalization, Phase 3
  • 11111: Recovery.Equalization, Done
rx_par_err

Output

When asserted for a single cycle, indicates that a parity error was detected in a TLP at the input of the RX buffer. The SR-IOV bridge drives this signal to the Application Layer without taking any action. If this error occurs, you must reset the Hard IP because parity errors can leave the Hard IP in an unknown state.

tx_par_err[1:0]

Output

When asserted for a single cycle, indicates a parity error during TX TLP transmission. The SR-IOV bridge drives this signal to the Application Layer without taking any action. The following encodings are defined:

  • 2’b10: A parity error was detected by the TX Transaction Layer.
  • 2’b01: Some time later, the parity error is detected by the TX Data Link Layer which drives 2’b01 to indicate the error. Intel recommends resetting the Arria® 10 Hard IP for PCI Express when this error is detected. Contact Intel if resetting becomes unworkable.
Note: Not all simulation models assert the Transaction Layer error bit in conjunction with the Data Link Layer error bit.
ko_cpl_spc_data[11:0]

Output

The Application Layer can use this signal to build circuitry to prevent RX buffer overflow for completion data. Endpoints must advertise infinite space for completion data; however, RX buffer space is finite. ko_cpl_spc_data is a static signal that reflects the total number of 16 byte completion data units that can be stored in the completion RX buffer.

ko_cpl_spc_header]7:0]

Output

The Application Layer can use this signal to build circuitry to prevent RX buffer overflow for completion headers. Endpoints must advertise infinite space for completion headers; however, RX buffer space is finite. ko_cpl_spc_header is a static signal that indicates the total number of completion headers that can be stored in the RX buffer.

rxfc_cplbuf_ovf Output When asserted, indicates RX Posted Completion buffer overflow.
Related Information
4 Debug signals are not rigorously verified and should only be used to observe behavior. Debug signals should not be used to drive logic custom logic.
Level Two Title