Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide

Download PDF
ID 683686
Date 9/12/2024
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Getting Started with the SR-IOV Design Example 3. Parameter Settings 4. Physical Layout 5. Interfaces and Signal Descriptions 6. Registers 7. Reset and Clocks 8. Programming and Testing SR-IOV Bridge MSI Interrupts 9. Error Handling 10. IP Core Architecture 11. Design Implementation 12. Debugging 13. Document Revision History A. Transaction Layer Packet (TLP) Header Formats B. Arria® 10 Avalon-ST with SR-IOV Interface for PCIe Solutions User Guide Archive
1. Datasheet x
1.1. Arria® 10 Avalon® -ST Interface with SR-IOV for PCI Express* Datasheet 1.2. Release Information 1.3. Device Family Support 1.4. Debug Features 1.5. IP Core Verification 1.6. Performance and Resource Utilization 1.7. Recommended Speed Grades for SR-IOV Interface
1.1. Arria® 10 Avalon® -ST Interface with SR-IOV for PCI Express* Datasheet x
1.1.1. SR-IOV Features
1.5. IP Core Verification x
1.5.1. Compatibility Testing Environment
2. Getting Started with the SR-IOV Design Example x
2.1. Directory Structure for Arria® 10 SR-IOV Design Example 2.2. Design Components for the SR-IOV Design Example 2.3. Generating the SR-IOV Design Example 2.4. Compiling and Simulating the Design for SR-IOV
3. Parameter Settings x
3.1. Parameters 3.2. Arria® 10 Avalon-ST Settings 3.3. Arria® 10 SR-IOV System Settings 3.4. Base Address Register (BAR) Settings 3.5. SR-IOV Device Identification Registers 3.6. Arria® 10 Interrupt Capabilities 3.7. Physical Function TLP Processing Hints (TPH) 3.8. Address Translation Services (ATS) 3.9. PCI Express and PCI Capabilities Parameters 3.10. PHY Characteristics 3.11. Example Designs
3.9. PCI Express and PCI Capabilities Parameters x
3.9.1. PCI Express and PCI Capabilities 3.9.2. Error Reporting 3.9.3. Link Capabilities 3.9.4. Slot Capabilities 3.9.5. Power Management
4. Physical Layout x
4.1. Hard IP Block Placement In Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. Interfaces and Signal Descriptions x
5.1. Avalon-ST TX Interface 5.2. Component-Specific Avalon-ST Interface Signals 5.3. Avalon-ST RX Interface 5.4. BAR Hit Signals 5.5. Configuration Status Interface 5.6. Clock Signals 5.7. Function-Level Reset (FLR) Interface 5.8. SR-IOV Interrupt Interface 5.9. Configuration Extension Bus (CEB) Interface 5.10. Implementing MSI-X Interrupts 5.11. Control Shadow Interface 5.12. Local Management Interface (LMI) Signals 5.13. Reset, Status, and Link Training Signals 5.14. Hard IP Reconfiguration Interface 5.15. Serial Data Signals 5.16. Test Signals 5.17. PIPE Interface Signals 5.18. Arria® 10 Development Kit Conduit Interface
5.9. Configuration Extension Bus (CEB) Interface x
5.9.1. Vital Product Data (VPD) Capability
5.9.1. Vital Product Data (VPD) Capability x
5.9.1.1. Determine the Pointer Address of an External Capability Register 5.9.1.2. VPD Capability Implementation
6. Registers x
6.1. Addresses for Physical and Virtual Functions 6.2. Correspondence between Configuration Space Registers and the PCIe Specification 6.3. PCI and PCI Express Configuration Space Registers 6.4. MSI Registers 6.5. MSI-X Capability Structure 6.6. Power Management Capability Structure 6.7. PCI Express Capability Structure 6.8. Advanced Error Reporting (AER) Enhanced Capability Header Register 6.9. Uncorrectable Error Status Register 6.10. Uncorrectable Error Mask Register 6.11. Uncorrectable Error Severity Register 6.12. Correctable Error Status Register 6.13. Correctable Error Mask Register 6.14. Advanced Error Capabilities and Control Register 6.15. Header Log Registers 0-3 6.16. SR-IOV Virtualization Extended Capabilities Registers 6.17. Virtual Function Registers
6.3. PCI and PCI Express Configuration Space Registers x
6.3.1. Type 0 Configuration Space Registers 6.3.2. PCI and PCI Express Configuration Space Register Content 6.3.3. Interrupt Line and Interrupt Pin Register
6.16. SR-IOV Virtualization Extended Capabilities Registers x
6.16.1. SR-IOV Virtualization Extended Capabilities Registers Address Map 6.16.2. ARI Enhanced Capability Header 6.16.3. SR-IOV Enhanced Capability Registers 6.16.4. Initial VFs and Total VFs Registers 6.16.5. VF Device ID Register 6.16.6. Page Size Registers 6.16.7. VF Base Address Registers (BARs) 0-5 6.16.8. Secondary PCI Express Extended Capability Header 6.16.9. Lane Status Registers 6.16.10. Transaction Processing Hints (TPH) Requester Enhanced Capability Header
7. Reset and Clocks x
7.1. Reset Sequence for Hard IP for PCI Express IP Core and Application Layer 7.2. Function Level Reset (FLR) 7.3. Clocks
7.3. Clocks x
7.3.1. Clock Domains 7.3.2. Clock Summary
7.3.1. Clock Domains x
7.3.1.1. coreclkout_hip 7.3.1.2. pld_clk
8. Programming and Testing SR-IOV Bridge MSI Interrupts x
8.1. Setting Up and Verifying MSI Interrupts 8.2. Masking MSI Interrupts 8.3. Dropping a Pending MSI Interrupt
9. Error Handling x
9.1. Physical Layer Errors 9.2. Data Link Layer Errors 9.3. Transaction Layer Errors 9.4. Error Reporting and Data Poisoning 9.5. Uncorrectable and Correctable Error Status Bits
10. IP Core Architecture x
10.1. PCI Express Protocol Stack 10.2. Data Link Layer 10.3. Physical Layer 10.4. Top-Level Interfaces 10.5. Arria® 10 Hard IP for PCI Express with Single-Root I/O Virtualization (SR-IOV)
10.4. Top-Level Interfaces x
10.4.1. Avalon-ST Interface 10.4.2. Clocks and Reset 10.4.3. Interrupts 10.4.4. PIPE
11. Design Implementation x
11.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 11.2. Recommended Reset Sequence to Avoid Link Training Issues 11.3. SDC Timing Constraints
12. Debugging x
12.1. Setting Up Simulation 12.2. Simulation Fails To Progress Beyond Polling.Active State 12.3. Hardware Bring-Up Issues 12.4. Link Training 12.5. Creating a Signal Tap Debug File to Match Your Design Hierarchy 12.6. Use Third-Party PCIe Analyzer 12.7. BIOS Enumeration Issues
12.1. Setting Up Simulation x
12.1.1. Changing Between Serial and PIPE Simulation 12.1.2. Using the PIPE Interface for Gen1 and Gen2 Variants 12.1.3. Viewing the Important PIPE Interface Signals 12.1.4. Disabling the Scrambler for Gen1 and Gen2 Simulations 12.1.5. Disabling 8B/10B Encoding and Decoding for Gen1 and Gen2 Simulations
13. Document Revision History x
13.1. Document Revision History for the Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
1. Datasheet
2. Getting Started with the SR-IOV Design Example
3. Parameter Settings
4. Physical Layout
5. Interfaces and Signal Descriptions
6. Registers
7. Reset and Clocks
8. Programming and Testing SR-IOV Bridge MSI Interrupts
9. Error Handling
10. IP Core Architecture
11. Design Implementation
12. Debugging
13. Document Revision History
A. Transaction Layer Packet (TLP) Header Formats
B. Arria® 10 Avalon-ST with SR-IOV Interface for PCIe Solutions User Guide Archive

5.8. SR-IOV Interrupt Interface

The SR-IOV Bridge supports MSI and MSI-X interrupts for both Physical and Virtual Functions. The Application Layer can use this interface to generate MSI or MSI-X interrupts from both PFs and VFs. The SR-IOV Bridge also supports legacy Interrupts for Physical Functions if you configure the core to support only PFs. To support only PFs, turn on Enable SR-IOV support on the SR-IOV System Settings tab of the component GUI. The Application Layer should select one of the three types of interrupts, depending on the support provided by the platform and the software drivers. Ground the input pins for the unused interrupt types.

This interface also includes signals to set and clear the individual bits in the MSI Pending Bit Register.

Table 29.  MSI Interrupts

Signal

Direction

Description

app_msi_req

Input

When asserted, the Application Layer is requesting that an MSI interrupt be sent. Assertion causes an MSI posted write TLP to be generated based on the MSI configuration register values of the specified Function, and the setting of the app_msi_tc and app_msi_num inputs.

app_msi_req_fn[1:0]

Input

Specifies the PF generating the MSI interrupt. It must be set to the interrupting PF number when asserting app_msi_req.

app_msi_ack

Output

Ack for MSI interrupts. When asserted, indicates that Hard IP has sent an MSI posted write TLP in response app_msi_req . The Application Layer must wait for app_msi_ack after asserting app_msi_req. The Application Layer must de-assert app_msi_req for at least 1 cycle before signaling a new MSI interrupt.

app_msi_addr_pf[64*<n>-1:0]

Output

Driven by the MSI address registers of the PFs. <n> is the number of PFs.
app_msi_data_pf[16<n>-1:0]

Output

Driven by the MSI Data Registers of the PFs, . <n>= the number of PFs.

app_msi_enable_pf[<n>-1:0]

Output

Driven by the MSI Enable bit of the MSI Control Registers of the PFs.

app_msi_mask_pf[32<n>-1:0]

Output

The MSI Mask Bits of the MSI Capability Structure drive app_msi_mask_pf. This mask allows software to disable or defer message sending on a per-vector basis. app_msi_mask_pf[31:0] mask vectors for PF0.app_msi_mask_pf[63:32] mask vectors for PF1.

app_msi_multi_msg_enable_pf[3*<n>-1:0]

Output

Defines the number of interrupt vectors enabled for each PF. The following encodings are defined:

  • 3'b000: 1 vector
  • 3'b001: 2 vectors
  • 3'b010: 4 vectors
  • 3'b100: 16 vectors
  • 3'b101: 32 vectors

The MSI Multiple Message Enable field of the MSI Control Register of PF0 drives app_msi_multi_msg_enable_pf[2:0]. The MSI Multiple Message Enable field of the MSI Control Register of PF1 drives app_msi_multi_msg_enable_pf[5:3], and so on.

app_msi_num[4:0]

Input

Identifies the MSI interrupt type to be generated. Provides the low-order message data bits to be sent in the message data field of MSI messages. Only bits that are enabled by the apply.

app_msi_pending_bit_write_data

Input

Writes the MSI Pending Bit Register of the specified PF when app_msi_pending_bit_write_en is asserted. app_msi_num[4:0] specifies the bit to be written. For more information about the MSI Pending Bit Array (PBA), refer to Section 6.8.1.7 Mask Bits for MSI (Optional) in the PCI Local Bus Specification, Revision 3.0. Refer to Timing Diagram for MSI Pending Bit Write Operation below.

app_msi_pending_bit_write_en

Input

Writes a 0 or 1 into selected bit position in the MSI Pending Bit Register. app_msi_num[4:0] specifies the bit to be written. msi_pending_bit_write_data specifies the data to be written (0 or 1). app_msi_req_fn specifies the function number. This input must be asserted for one cycle to perform the write operation.

msi_pending_bit_write_en cannot be asserted when app_msi_req is high. Refer to Timing Diagram for MSI Pending Bit Write Operation below.

app_msi_pending_pf[32*<n>-1:0]

Output

The MSI Data Registers of the PFs drive msi_pending_pf. <n> is the number of PFs.

app_msi_tc[2:0]

Input

Specifies the traffic class to be used to send the MSI or MSI-X posted write TLP. Must be valid when app_msi_req is asserted.

app_msi_status[1:0] Output

Specifies the status of an MSI request. Valid when app_msi_ack is asserted. The following encodings are defined:

  • 2'b00: MSI message sent
  • 2'b01: MSI message is pending and not sent upstream because the MSI mask was set. And, the Pending bit was set for the MSI number.
  • 2'b10: Requested aborted because of invalid parameters. Or, request aborted because the MSI Enabled bit was not set in the function's MSI Capability structure.
  • 2'b11: Reserved.
Figure 25. Timing Diagram for MSI Interrupt Generation
Figure 26. Timing Diagram for MSI Pending Bit Write Operation The MSI Pending Bit Write Operation aborts the pending MSI interrupt.
Table 30.   MSI-X Interrupts

Signal

Direction

Description

app_msix_req

Input

When asserted, the Application Layer is requesting that an MSI-X interrupt be sent. Assertion causes an MSI-X posted write TLP to be generated. The MSI-X TLP uses data from app_msi_req_pf_num, app_msi_req_ vf_num, app_msi_req_vf_active, app_msix_addr, app_msix_data, and app_msi_tc inputs. Refer to Timing Diagram for MSI-X Interrupt Generation below.

app_msix_ack

Output

Ack for MSI-X interrupts. When asserted, indicates that Hard IP has sent an MSI-X posted write TLP in response app_msix_req . The Application Layer must wait for after asserting app_msix_req. The Application Layer must de-assert app_msix_req for at least 1 cycle before signaling a new MSI interrupt.

app_msix_addr[63:0]

Input

The Application Layer drives the address for the MSI-X posted write TLP on this input. Driven in the same cycle as app_msix_req.

app_msix_data[31:0]

Input

The Application Layer drives app_msix_data[31:0] for the MSI-X posted write TLP. Driven in the same cycle as app_msix_req.

app_msix_enable_pf[<n>-1:0] Output Driven by the MSIX Enable bit of the MSIX Control Register of the PFs.

app_msix_pf_num[1:0]

Output

Identifies the Physical Function generating the MSI-X interrupt. It must be set to the interrupting Function number when asserting app_msix_req.

When the targeted Function is a VF, this input specifies the PF Number to which the VF is attached.
app_msix_vf_active Input Specifies that the Function generating the MSI-X interrupt is a Virtual Function. If this input is asserted, the user must provide the VF number offset of the VF generating the interrupt on app_msix_vf_num.
app_msix_vf_num[10:0] Input When app_msix_vf_active is asserted, this input identifies the VF number offset for the VF generating the interrupt. Its value ranges from 0-<n>-1, where i<n>s the number of VFs in the set of VFs attached to the associated PF.
app_msix_tc[2:0] Input Specifies the traffic class of the MSI-X posted write TLP. It must be valid when app_msix_req is asserted.
app_msix_ack Output Acknowledgment for MSI-X interrupts. A pulse on this output indicates that an MSI-X posted write TLP has been sent in response to the assertion of the app_msix_req input. The user application must wait for the acknowledgment after asserting app_msix_req, and must de-assert app_msix_req for at least one cycle before signaling a new MSI-X interrupt.
     
app_msix_err

Output

Signals an error during the execution of an MSI-X request. Valid when app_msix_ack is asserted. The following encodings are defined:

  • 1b'0: MSI-X message sent
  • 1b'1: Error detected during execution of the MSI-X request. No message sent. The following errors may occur:
    • The function number is invalid
    • The MSI-X Enable bit for the function was not set
    • The MSI-X Function Mask was not set

app_msix_fn_mask_pf[<n>-1:0]

Output

Driven by the MSI-X Function Mask bit of the MSI-X Control Register of the PFs.

     
Figure 27. Timing Diagram for MSI-X Interrupt Generation
Table 31.  Legacy Interrupts

Signal

Direction

Description

app_int_pf_sts[<n>-1:0]

Input

The Application Layer uses this signal to generate a legacy INT<n> interrupt. PF<n> drives app_int_pf_sts[<n>-1:0] . The Hard IP sends a INTx_Assert message upstream to the Root Complex in response to a low-to- high transition. The Hard IP sends a INTX_Deassert in response to a high-to-low transition. The INTX_Deassert message is only sent if a previous INTx_Assert message was sent. When multiple Functions share the same INTx pin, only a single INTx_Assert message is sent if more than one interrupt sharing the same INTx pin are active at the same time.

This input has no effect if the INT<n>Disable bit in the PCI Command Register of the interrupting function is set to 1.

     

app_int_sts_fn

Input

Identifies the function generating the legacy interrupt. When app_int_sts_fn = 0, specifies status for PF0. When app_int_sts_fn = 1, specifies status for PF1.

app_intx_disable[<n>-1:0]

Output

This output is driven by the INT<x>Disable bit of the PCI Command Register of PF.

Related Information
Level Two Title