Arria® 10 or Cyclone® 10 GX Avalon® Memory-Mapped (Avalon-MM) DMA Interface for PCI Express* Solutions User Guide

Download PDF
ID 683425
Date 9/10/2024
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Getting Started with the Avalon-MM DMA 3. Parameter Settings 4. Physical Layout 5. IP Core Interfaces 6. Registers 7. Reset and Clocks 8. Error Handling 9. PCI Express Protocol Stack 10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express 11. Design Implementation A. Transaction Layer Packet (TLP) Header Formats B. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe Solutions User Guide Archive C. Document Revision History
1. Datasheet x
1.1. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe* Datasheet 1.2. Features 1.3. Comparison of Avalon-ST, Avalon-MM and Avalon-MM with DMA Interfaces 1.4. Release Information 1.5. Device Family Support 1.6. Design Examples 1.7. IP Core Verification 1.8. Resource Utilization 1.9. Recommended Speed Grades 1.10. Creating a Design for PCI Express
1.7. IP Core Verification x
1.7.1. Compatibility Testing Environment
2. Getting Started with the Avalon-MM DMA x
2.1. Understanding the Avalon-MM DMA Ports 2.2. Generating the Testbench 2.3. Simulating the Example Design in ModelSim* 2.4. Running a Gate-Level Simulation 2.5. Generating Synthesis Files 2.6. Compiling the Design 2.7. Creating a Quartus® Prime Project
2.2. Generating the Testbench x
2.2.1. Understanding the Simulation Generated Files 2.2.2. Understanding Simulation Log File Generation
3. Parameter Settings x
3.1. Parameters 3.2. Arria® 10 or Cyclone® 10 GX Avalon-MM Settings 3.3. Base Address Register (BAR) Settings 3.4. Device Identification Registers 3.5. PCI Express and PCI Capabilities Parameters 3.6. Configuration, Debug, and Extension Options 3.7. PHY Characteristics 3.8. Example Designs
3.1. Parameters x
3.1.1. RX Buffer Allocation Selections Available by Interface Type
3.5. PCI Express and PCI Capabilities Parameters x
3.5.1. Device Capabilities 3.5.2. Error Reporting 3.5.3. Link Capabilities 3.5.4. MSI and MSI-X Capabilities 3.5.5. Slot Capabilities 3.5.6. Power Management 3.5.7. Vendor Specific Extended Capability (VSEC)
4. Physical Layout x
4.1. Hard IP Block Placement In Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. IP Core Interfaces x
5.1. Arria® 10 or Cyclone® 10 GX DMA Avalon-MM DMA Interface to the Application Layer 5.2. Clock Signals 5.3. Reset, Status, and Link Training Signals 5.4. MSI Interrupts for Endpoints 5.5. Hard IP Reconfiguration Interface 5.6. Physical Layer Interface Signals 5.7. Test Signals 5.8. Arria® 10 Development Kit Conduit Interface
5.1. Arria® 10 or Cyclone® 10 GX DMA Avalon-MM DMA Interface to the Application Layer x
5.1.1. Avalon-MM DMA Interfaces when Descriptor Controller Is Internally Instantiated 5.1.2. Read Data Mover 5.1.3. Write DMA Avalon-MM Master Port 5.1.4. RX Master Module 5.1.5. Non-Bursing Slave Module 5.1.6. 32-Bit Control Register Access (CRA) Slave Signals 5.1.7. Avalon-ST Descriptor Control Interface when Instantiated Separately 5.1.8. Descriptor Controller Interfaces when Instantiated Internally
5.1.7. Avalon-ST Descriptor Control Interface when Instantiated Separately x
5.1.7.1. Avalon-ST Descriptor Status Interface 5.1.7.2. Avalon® -ST Descriptor Status Sources
5.1.8. Descriptor Controller Interfaces when Instantiated Internally x
5.1.8.1. Read Descriptor Controller Avalon-MM Master interface 5.1.8.2. Write Descriptor Controller Avalon-MM Master Interface 5.1.8.3. Read Descriptor Table Avalon-MM Slave Interface 5.1.8.4. Write Descriptor Table Avalon-MM Slave Interface
5.6. Physical Layer Interface Signals x
5.6.1. Serial Data Signals 5.6.2. PIPE Interface Signals
6. Registers x
6.1. Correspondence between Configuration Space Registers and the PCIe Specification 6.2. Type 0 Configuration Space Registers 6.3. Type 1 Configuration Space Registers 6.4. PCI Express Capability Structures 6.5. Intel-Defined VSEC Registers 6.6. Advanced Error Reporting Capability 6.7. DMA Descriptor Controller Registers 6.8. Control Register Access (CRA) Avalon-MM Slave Port
6.5. Intel-Defined VSEC Registers x
6.5.1. CvP Registers
6.6. Advanced Error Reporting Capability x
6.6.1. Uncorrectable Internal Error Mask Register 6.6.2. Uncorrectable Internal Error Status Register 6.6.3. Correctable Internal Error Mask Register 6.6.4. Correctable Internal Error Status Register
6.7. DMA Descriptor Controller Registers x
6.7.1. Read DMA Descriptor Controller Registers 6.7.2. Write DMA Descriptor Controller Registers 6.7.3. Read DMA and Write DMA Descriptor Format 6.7.4. Read DMA Example 6.7.5. Software Program for Simultaneous Read and Write DMA
7. Reset and Clocks x
7.1. Reset Sequence for Hard IP for PCI Express IP Core and Application Layer 7.2. Clocks
7.2. Clocks x
7.2.1. Clock Domains 7.2.2. Clock Summary
7.2.1. Clock Domains x
7.2.1.1. coreclkout_hip 7.2.1.2. pld_clk
8. Error Handling x
8.1. Physical Layer Errors 8.2. Data Link Layer Errors 8.3. Transaction Layer Errors 8.4. Error Reporting and Data Poisoning 8.5. Uncorrectable and Correctable Error Status Bits
9. PCI Express Protocol Stack x
9.1. Top-Level Interfaces 9.2. Data Link Layer 9.3. Physical Layer
9.1. Top-Level Interfaces x
9.1.1. Avalon-MM DMA Interface 9.1.2. Clocks and Reset 9.1.3. Interrupts 9.1.4. PIPE
10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express x
10.1. Understanding the Internal DMA Descriptor Controller 10.2. Understanding the External DMA Descriptor Controller
11. Design Implementation x
11.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 11.2. Recommended Reset Sequence to Avoid Link Training Issues 11.3. SDC Timing Constraints 11.4. Warnings Encountered When Using Narrow Avalon-MM Interfaces
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
C. Document Revision History x
C.1. Document Revision History for the Arria® 10 or Cyclone® 10 GX Avalon® Memory Mapped (Avalon-MM) DMA Interface for PCIe* Solutions User Guide
1. Datasheet
2. Getting Started with the Avalon-MM DMA
3. Parameter Settings
4. Physical Layout
5. IP Core Interfaces
6. Registers
7. Reset and Clocks
8. Error Handling
9. PCI Express Protocol Stack
10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express
11. Design Implementation
A. Transaction Layer Packet (TLP) Header Formats
B. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe Solutions User Guide Archive
C. Document Revision History

6.7.4. Read DMA Example

This example moves three data blocks from the system memory to the Avalon-MM address space. Host software running on an embedded CPU allocates the memory and creates the descriptor table in system memory.

This example uses the addresses in the Platform Designer design example, ep_g3x8_avmm256_integrated.qsys, available in the <install_dir>/ ip/altera/altera_pcie/altera_pcie_<dev>_ed/example_design/<dev> directory.

The following figures illustrate the location and size of the data blocks in the PCIe and Avalon-MM address spaces and the descriptor table format. In this example, the value of RD_TABLE_SIZE is 127.

Figure 36. Data Blocks to Transfer from PCIe to Avalon-MM Address Space Using Read DMA

Assume the descriptor table includes 128 entries. The status table precedes a variable number of descriptors in memory. The Read and Write Status and Descriptor Tables are at the address specified in the RC Read Descriptor Base Register and RC Write Descriptor Base Register, respectively.

Figure 37. Descriptor Table Format
  1. Calculate the memory allocation required:
    1. Each entry in the status table is 4 bytes. The 128 entries require 512 bytes of memory.
    2. Each descriptor is 32 bytes. The three descriptors require 96 bytes of memory.
    The total memory allocation for the status and descriptor tables is 608 bytes.
  2. Allocate 608 bytes of memory in the PCI Express address space.
    The start address of the allocated memory in this example is 0xF000_0000. Program this address into the Root Complex Read Status and Descriptor Base Address Registers.
  3. Create the descriptor table in the PCI Express address space. Because the status table is stored before the descriptors, the first descriptor is stored at 0xF000_0000 + 0x200 = 0xF000_0200.
    1. Program 0x0000_0000 into the source address 0xF000_0204 in descriptor 0.
      This is the upper 32 bits of the source address.
    2. Program 0x1000_0000 into the source address 0xF000_0200 in descriptor 0.
      This is the lower 32 bits of the source address.
    3. Program 0x0000_0000 into the destination address 0xF000_020C in descriptor 0.
      This is the upper 32 bits of the destination address.
    4. Program 0x5000_0000 into the destination address 0xF000_0208 in descriptor 0.
      This is the lower 32 bits of the destination address.
      These four steps program the Avalon-MM destination address for the 64 KB block of memory into the Descriptor Table.
    5. Program 0x0000_4000 to 0xF000_0210 to transfer 16K dwords (64 KB), of data for descriptor ID 0.
  4. Repeat this procedure for the second data block:
    1. Program 0x0000_0000 to source address 0xF000_0224.
    2. Program 0x2000_0000 to source address 0xF000_0220.
    3. Program 0x0000_0000 to destination address 0xF000_022C.
    4. Program 0x0001_0000 to destination address 0xF000_0228.
    5. Program 0x0004_2000 to 0xF000_0230 to transfer 8K dwords (32 KB) of data for descriptor ID 1.
  5. Repeat this procedure for the third data block:
    1. Program 0x0000_0001 to source address 0xF000_0244.
    2. Program 0x2000_0000 to source address 0xF000_0240.
    3. Program 0x0000_0000 to destination address 0xF000_024C.
    4. Program 0x1000_0000 to destination address 0xF000_0248.
    5. Program 0x0008_1000 to 0xF000_0250 to transfer 4K dwords (16 KB) of data for descriptor ID 2.
    The following figure shows the values in the Descriptor Table after programming completes.
    Figure 38. Descriptor Table Format
  6. Program the DMA Descriptor Controller with the address of the status and descriptor table in the PCI Express system memory address space. When the DMA Descriptor Controller is internal, these registers are accessed through combined BAR0 and BAR1 because this example uses 64-bit addresses. The DMA read control registers start at offset 0x0000. The Write DMA control registers start at offset 0x0100
    1. Program 0x0000_0000 to offset 0x0000_0004.
      This is the upper 32 bits of the PCIe system memory where the status and descriptor table is stored.
    2. Program 0xF000_0000 to offset 0x0000_0000.
      This is the lower 32 bits of the address in PCIe memory that stores the status and descriptor tables. The Read DMA automatically adds an offset of 0x200 to this value to start the copy of the descriptors which follow the status table in memory.
  7. Program the DMA Descriptor Controller with the on-chip FIFO address. This is the address to which the Descriptor Controller copies the status and descriptor table.
    1. Program 0x0000_0000 to offset 0x0000_000C
      This is the upper 32 bits of the on-chip FIFO address in the Avalon-MM address domain.
    2. Program 0x0100_0000 to offset 0x0000_0008.
      This is the lower 32 bits of the on-chip FIFO address. This is address of the internal on-chip FIFO that is a part of the Descriptor Controller as seen by the RX Master.
      Figure 39. Address of the On-Chip FIFO
  8. Program the Descriptor Controller RD_DMA_LAST_PTR register.
    This step starts the DMA. It also specifies the status dword to be updated when the three descriptors complete.

    • To update a single done bit for the final descriptor, program 0x2 to offset 0x0000_0010. The Descriptor Controller processes all three descriptors and writes the done bit to 0xF000_0008 of the status table.

    • To update the done bits for all three descriptors, program address 0x0000_0010 RD_DMA_LAST_PTR three times with the values 0, 1, and 2. The Descriptor Controller sets the done bits for addresses 0xF000_0000, 0xF000_0004, and 0xF000_0008. If the system returns Read Completions out-of-order, the Descriptor Controller may complete descriptors out of order. In such systems, you must use this method of requesting done status for each descriptor. Software must check for done status for every descriptor.
Related Information
Level Two Title