Arria® 10 or Cyclone® 10 GX Avalon® Memory-Mapped (Avalon-MM) DMA Interface for PCI Express* Solutions User Guide

Download PDF
ID 683425
Date 9/10/2024
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Getting Started with the Avalon-MM DMA 3. Parameter Settings 4. Physical Layout 5. IP Core Interfaces 6. Registers 7. Reset and Clocks 8. Error Handling 9. PCI Express Protocol Stack 10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express 11. Design Implementation A. Transaction Layer Packet (TLP) Header Formats B. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe Solutions User Guide Archive C. Document Revision History
1. Datasheet x
1.1. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe* Datasheet 1.2. Features 1.3. Comparison of Avalon-ST, Avalon-MM and Avalon-MM with DMA Interfaces 1.4. Release Information 1.5. Device Family Support 1.6. Design Examples 1.7. IP Core Verification 1.8. Resource Utilization 1.9. Recommended Speed Grades 1.10. Creating a Design for PCI Express
1.7. IP Core Verification x
1.7.1. Compatibility Testing Environment
2. Getting Started with the Avalon-MM DMA x
2.1. Understanding the Avalon-MM DMA Ports 2.2. Generating the Testbench 2.3. Simulating the Example Design in ModelSim* 2.4. Running a Gate-Level Simulation 2.5. Generating Synthesis Files 2.6. Compiling the Design 2.7. Creating a Quartus® Prime Project
2.2. Generating the Testbench x
2.2.1. Understanding the Simulation Generated Files 2.2.2. Understanding Simulation Log File Generation
3. Parameter Settings x
3.1. Parameters 3.2. Arria® 10 or Cyclone® 10 GX Avalon-MM Settings 3.3. Base Address Register (BAR) Settings 3.4. Device Identification Registers 3.5. PCI Express and PCI Capabilities Parameters 3.6. Configuration, Debug, and Extension Options 3.7. PHY Characteristics 3.8. Example Designs
3.1. Parameters x
3.1.1. RX Buffer Allocation Selections Available by Interface Type
3.5. PCI Express and PCI Capabilities Parameters x
3.5.1. Device Capabilities 3.5.2. Error Reporting 3.5.3. Link Capabilities 3.5.4. MSI and MSI-X Capabilities 3.5.5. Slot Capabilities 3.5.6. Power Management 3.5.7. Vendor Specific Extended Capability (VSEC)
4. Physical Layout x
4.1. Hard IP Block Placement In Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. IP Core Interfaces x
5.1. Arria® 10 or Cyclone® 10 GX DMA Avalon-MM DMA Interface to the Application Layer 5.2. Clock Signals 5.3. Reset, Status, and Link Training Signals 5.4. MSI Interrupts for Endpoints 5.5. Hard IP Reconfiguration Interface 5.6. Physical Layer Interface Signals 5.7. Test Signals 5.8. Arria® 10 Development Kit Conduit Interface
5.1. Arria® 10 or Cyclone® 10 GX DMA Avalon-MM DMA Interface to the Application Layer x
5.1.1. Avalon-MM DMA Interfaces when Descriptor Controller Is Internally Instantiated 5.1.2. Read Data Mover 5.1.3. Write DMA Avalon-MM Master Port 5.1.4. RX Master Module 5.1.5. Non-Bursing Slave Module 5.1.6. 32-Bit Control Register Access (CRA) Slave Signals 5.1.7. Avalon-ST Descriptor Control Interface when Instantiated Separately 5.1.8. Descriptor Controller Interfaces when Instantiated Internally
5.1.7. Avalon-ST Descriptor Control Interface when Instantiated Separately x
5.1.7.1. Avalon-ST Descriptor Status Interface 5.1.7.2. Avalon® -ST Descriptor Status Sources
5.1.8. Descriptor Controller Interfaces when Instantiated Internally x
5.1.8.1. Read Descriptor Controller Avalon-MM Master interface 5.1.8.2. Write Descriptor Controller Avalon-MM Master Interface 5.1.8.3. Read Descriptor Table Avalon-MM Slave Interface 5.1.8.4. Write Descriptor Table Avalon-MM Slave Interface
5.6. Physical Layer Interface Signals x
5.6.1. Serial Data Signals 5.6.2. PIPE Interface Signals
6. Registers x
6.1. Correspondence between Configuration Space Registers and the PCIe Specification 6.2. Type 0 Configuration Space Registers 6.3. Type 1 Configuration Space Registers 6.4. PCI Express Capability Structures 6.5. Intel-Defined VSEC Registers 6.6. Advanced Error Reporting Capability 6.7. DMA Descriptor Controller Registers 6.8. Control Register Access (CRA) Avalon-MM Slave Port
6.5. Intel-Defined VSEC Registers x
6.5.1. CvP Registers
6.6. Advanced Error Reporting Capability x
6.6.1. Uncorrectable Internal Error Mask Register 6.6.2. Uncorrectable Internal Error Status Register 6.6.3. Correctable Internal Error Mask Register 6.6.4. Correctable Internal Error Status Register
6.7. DMA Descriptor Controller Registers x
6.7.1. Read DMA Descriptor Controller Registers 6.7.2. Write DMA Descriptor Controller Registers 6.7.3. Read DMA and Write DMA Descriptor Format 6.7.4. Read DMA Example 6.7.5. Software Program for Simultaneous Read and Write DMA
7. Reset and Clocks x
7.1. Reset Sequence for Hard IP for PCI Express IP Core and Application Layer 7.2. Clocks
7.2. Clocks x
7.2.1. Clock Domains 7.2.2. Clock Summary
7.2.1. Clock Domains x
7.2.1.1. coreclkout_hip 7.2.1.2. pld_clk
8. Error Handling x
8.1. Physical Layer Errors 8.2. Data Link Layer Errors 8.3. Transaction Layer Errors 8.4. Error Reporting and Data Poisoning 8.5. Uncorrectable and Correctable Error Status Bits
9. PCI Express Protocol Stack x
9.1. Top-Level Interfaces 9.2. Data Link Layer 9.3. Physical Layer
9.1. Top-Level Interfaces x
9.1.1. Avalon-MM DMA Interface 9.1.2. Clocks and Reset 9.1.3. Interrupts 9.1.4. PIPE
10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express x
10.1. Understanding the Internal DMA Descriptor Controller 10.2. Understanding the External DMA Descriptor Controller
11. Design Implementation x
11.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 11.2. Recommended Reset Sequence to Avoid Link Training Issues 11.3. SDC Timing Constraints 11.4. Warnings Encountered When Using Narrow Avalon-MM Interfaces
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
C. Document Revision History x
C.1. Document Revision History for the Arria® 10 or Cyclone® 10 GX Avalon® Memory Mapped (Avalon-MM) DMA Interface for PCIe* Solutions User Guide
1. Datasheet
2. Getting Started with the Avalon-MM DMA
3. Parameter Settings
4. Physical Layout
5. IP Core Interfaces
6. Registers
7. Reset and Clocks
8. Error Handling
9. PCI Express Protocol Stack
10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express
11. Design Implementation
A. Transaction Layer Packet (TLP) Header Formats
B. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe Solutions User Guide Archive
C. Document Revision History

6.7. DMA Descriptor Controller Registers

The DMA Descriptor Controller manages Read and Write DMA operations. The DMA Descriptor Controller is available for use with Endpoint variations. The Descriptor Controller supports up to 128 descriptors each for Read and Write Data Movers. Read and Write are from the perspective of the FPGA. A read is from PCIe* address space to the FPGA Avalon® -MM address space. A write is to PCIe* address space from the FPGA Avalon® -MM space.

You program the Descriptor Controller internal registers with the location and size of the descriptor table residing in the PCIe* address space. The DMA Descriptor Controller instructs the Read Data Mover to copy the table to its own internal FIFO. When the DMA Descriptor Controller is instantiated as a separate component, it drives table entries on the RdDmaRxData_i[159:0] and WrDmaRxData_i[159:0] buses. When the DMA Descriptor Controller is embedded inside the Avalon-MM DMA bridge, it drives this information on internal buses. .

The Read Data Mover transfers data from the PCIe address space to Avalon-MM address space. It issues memory read TLPs on the PCIe link. It writes the data returned to a location in the Avalon® -MM address space. The source address is the address for the data in the PCIe address space. The destination address is in the Avalon-MM address space.

The Write Data Mover reads data from the Avalon-MM address space and writes to the PCIe address space. It issues memory write TLPs on the PCIe link. The source address is in the Avalon-MM address space. The destination address is in the PCIe address space.

The DMA Descriptor Controller records the completion status for read and write descriptors in separate status tables. Each table has 128 consecutive DWORD entries that correspond to the 128 descriptors. The actual descriptors are stored immediately after the status entries at offset 0x200 from the values programmed into the RC Read Descriptor Base and RC Write Descriptor Base registers. The status and descriptor table must be located on a 32-byte boundary in Root Complex memory.

The Descriptor Controller writes a 1 to the done bit of the status DWORD to indicate successful completion. The Descriptor Controller also sends an MSI interrupt for the final descriptor. After receiving this MSI, host software can poll the done bit to determine status. The status table precedes the descriptor table in memory. The Descriptor Controller does not write the done bit nor send an MSI as each descriptor completes. It only writes the done bit or sends an MSI for the descriptor whose ID is stored in the RD_DMA_LAST PTR or WR_DMA_LAST_PTR registers. The Descriptor Controller supports out-of-order completions. Consequently, it is possible for the done bit to be set before all descriptors have completed.

Note: The following example clarifies this programming model. If 128 descriptors are specified and all of them execute, then descriptor ID 127 is written to the RD_DMA_LAST_PTR or WR_DMA_LAST_PTR register to start the DMA. The DMA Descriptor Controller only writes the done bit when descriptor 127 completes. To get intermediate status updates, host software should write multiple IDs into the last pointer register. For example, to get an intermediate status update when half of the 128 read descriptors have completed, host software should complete the following sequence:
  1. Program the RD_DMA_LAST_PTR = 63.
  2. Program the RD_DMA_LAST_PTR = 127.
  3. Poll the status DWORD for read descriptor 63.
  4. Poll the status DWORD for read descriptor 127.

Many commercial system Root Ports return out-of-order Read Completions based on optimized accesses to host memory channels. Consequently, the done status stored for descriptor <n> does not necessarily mean that descriptors <n-1> and <n -2> have also completed. You must request the completion status for every descriptor by writing the descriptor ID for every descriptor to RD_DMA_LAST_PTR or WR_DMA_LAST_PTR.

Note: Because the DMA Descriptor Controller uses FIFOs to store descriptor table entries, you cannot reprogram the DMA Descriptor Controller once it begins the transfers specified in the descriptor table.

Section Content
Read DMA Descriptor Controller Registers
Write DMA Descriptor Controller Registers
Read DMA and Write DMA Descriptor Format
Read DMA Example
Software Program for Simultaneous Read and Write DMA

Level Two Title