Arria® 10 or Cyclone® 10 GX Avalon® Memory-Mapped (Avalon-MM) DMA Interface for PCI Express* Solutions User Guide

Download PDF
ID 683425
Date 9/10/2024
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Getting Started with the Avalon-MM DMA 3. Parameter Settings 4. Physical Layout 5. IP Core Interfaces 6. Registers 7. Reset and Clocks 8. Error Handling 9. PCI Express Protocol Stack 10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express 11. Design Implementation A. Transaction Layer Packet (TLP) Header Formats B. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe Solutions User Guide Archive C. Document Revision History
1. Datasheet x
1.1. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe* Datasheet 1.2. Features 1.3. Comparison of Avalon-ST, Avalon-MM and Avalon-MM with DMA Interfaces 1.4. Release Information 1.5. Device Family Support 1.6. Design Examples 1.7. IP Core Verification 1.8. Resource Utilization 1.9. Recommended Speed Grades 1.10. Creating a Design for PCI Express
1.7. IP Core Verification x
1.7.1. Compatibility Testing Environment
2. Getting Started with the Avalon-MM DMA x
2.1. Understanding the Avalon-MM DMA Ports 2.2. Generating the Testbench 2.3. Simulating the Example Design in ModelSim* 2.4. Running a Gate-Level Simulation 2.5. Generating Synthesis Files 2.6. Compiling the Design 2.7. Creating a Quartus® Prime Project
2.2. Generating the Testbench x
2.2.1. Understanding the Simulation Generated Files 2.2.2. Understanding Simulation Log File Generation
3. Parameter Settings x
3.1. Parameters 3.2. Arria® 10 or Cyclone® 10 GX Avalon-MM Settings 3.3. Base Address Register (BAR) Settings 3.4. Device Identification Registers 3.5. PCI Express and PCI Capabilities Parameters 3.6. Configuration, Debug, and Extension Options 3.7. PHY Characteristics 3.8. Example Designs
3.1. Parameters x
3.1.1. RX Buffer Allocation Selections Available by Interface Type
3.5. PCI Express and PCI Capabilities Parameters x
3.5.1. Device Capabilities 3.5.2. Error Reporting 3.5.3. Link Capabilities 3.5.4. MSI and MSI-X Capabilities 3.5.5. Slot Capabilities 3.5.6. Power Management 3.5.7. Vendor Specific Extended Capability (VSEC)
4. Physical Layout x
4.1. Hard IP Block Placement In Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. IP Core Interfaces x
5.1. Arria® 10 or Cyclone® 10 GX DMA Avalon-MM DMA Interface to the Application Layer 5.2. Clock Signals 5.3. Reset, Status, and Link Training Signals 5.4. MSI Interrupts for Endpoints 5.5. Hard IP Reconfiguration Interface 5.6. Physical Layer Interface Signals 5.7. Test Signals 5.8. Arria® 10 Development Kit Conduit Interface
5.1. Arria® 10 or Cyclone® 10 GX DMA Avalon-MM DMA Interface to the Application Layer x
5.1.1. Avalon-MM DMA Interfaces when Descriptor Controller Is Internally Instantiated 5.1.2. Read Data Mover 5.1.3. Write DMA Avalon-MM Master Port 5.1.4. RX Master Module 5.1.5. Non-Bursing Slave Module 5.1.6. 32-Bit Control Register Access (CRA) Slave Signals 5.1.7. Avalon-ST Descriptor Control Interface when Instantiated Separately 5.1.8. Descriptor Controller Interfaces when Instantiated Internally
5.1.7. Avalon-ST Descriptor Control Interface when Instantiated Separately x
5.1.7.1. Avalon-ST Descriptor Status Interface 5.1.7.2. Avalon® -ST Descriptor Status Sources
5.1.8. Descriptor Controller Interfaces when Instantiated Internally x
5.1.8.1. Read Descriptor Controller Avalon-MM Master interface 5.1.8.2. Write Descriptor Controller Avalon-MM Master Interface 5.1.8.3. Read Descriptor Table Avalon-MM Slave Interface 5.1.8.4. Write Descriptor Table Avalon-MM Slave Interface
5.6. Physical Layer Interface Signals x
5.6.1. Serial Data Signals 5.6.2. PIPE Interface Signals
6. Registers x
6.1. Correspondence between Configuration Space Registers and the PCIe Specification 6.2. Type 0 Configuration Space Registers 6.3. Type 1 Configuration Space Registers 6.4. PCI Express Capability Structures 6.5. Intel-Defined VSEC Registers 6.6. Advanced Error Reporting Capability 6.7. DMA Descriptor Controller Registers 6.8. Control Register Access (CRA) Avalon-MM Slave Port
6.5. Intel-Defined VSEC Registers x
6.5.1. CvP Registers
6.6. Advanced Error Reporting Capability x
6.6.1. Uncorrectable Internal Error Mask Register 6.6.2. Uncorrectable Internal Error Status Register 6.6.3. Correctable Internal Error Mask Register 6.6.4. Correctable Internal Error Status Register
6.7. DMA Descriptor Controller Registers x
6.7.1. Read DMA Descriptor Controller Registers 6.7.2. Write DMA Descriptor Controller Registers 6.7.3. Read DMA and Write DMA Descriptor Format 6.7.4. Read DMA Example 6.7.5. Software Program for Simultaneous Read and Write DMA
7. Reset and Clocks x
7.1. Reset Sequence for Hard IP for PCI Express IP Core and Application Layer 7.2. Clocks
7.2. Clocks x
7.2.1. Clock Domains 7.2.2. Clock Summary
7.2.1. Clock Domains x
7.2.1.1. coreclkout_hip 7.2.1.2. pld_clk
8. Error Handling x
8.1. Physical Layer Errors 8.2. Data Link Layer Errors 8.3. Transaction Layer Errors 8.4. Error Reporting and Data Poisoning 8.5. Uncorrectable and Correctable Error Status Bits
9. PCI Express Protocol Stack x
9.1. Top-Level Interfaces 9.2. Data Link Layer 9.3. Physical Layer
9.1. Top-Level Interfaces x
9.1.1. Avalon-MM DMA Interface 9.1.2. Clocks and Reset 9.1.3. Interrupts 9.1.4. PIPE
10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express x
10.1. Understanding the Internal DMA Descriptor Controller 10.2. Understanding the External DMA Descriptor Controller
11. Design Implementation x
11.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 11.2. Recommended Reset Sequence to Avoid Link Training Issues 11.3. SDC Timing Constraints 11.4. Warnings Encountered When Using Narrow Avalon-MM Interfaces
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
C. Document Revision History x
C.1. Document Revision History for the Arria® 10 or Cyclone® 10 GX Avalon® Memory Mapped (Avalon-MM) DMA Interface for PCIe* Solutions User Guide
1. Datasheet
2. Getting Started with the Avalon-MM DMA
3. Parameter Settings
4. Physical Layout
5. IP Core Interfaces
6. Registers
7. Reset and Clocks
8. Error Handling
9. PCI Express Protocol Stack
10. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA for PCI Express
11. Design Implementation
A. Transaction Layer Packet (TLP) Header Formats
B. Arria® 10 or Cyclone® 10 GX Avalon-MM DMA Interface for PCIe Solutions User Guide Archive
C. Document Revision History

C.1. Document Revision History for the Arria® 10 or Cyclone® 10 GX Avalon® Memory Mapped (Avalon-MM) DMA Interface for PCIe* Solutions User Guide

Date

Version

Changes Made

2024.09.10 18.0.1 Added Notes with reset recommendations to the Reset and Clocks section.
2021.06.03 18.0.1 Mentioned in the Features section that this IP supports the Separate Reference Clock No Spread Spectrum (SRNS) architecture and not the Separate Reference Clock with Independent Spread Spectrum (SRIS) architecture.
2019.12.23 18.0.1 Changed the name of the 1A state of the ltssmstate signals to Recovery.Speed to follow the PCIe Specifications.
2019.05.23 18.0.1 Added a note clarifying that the 24-bit Class Code register is divided into three 8-bit fields: Base Class Code, Sub-Class Code and Programming Interface.
2019.04.30 18.0.1 Updated Table 3 to show that the Avalon-MM DMA feature is not supported in Root Port mode.
2018.08.28 18.0.1 Added the step to invoke vsim to the instructions for simulating the example design in ModelSim.
2018.06.15 18.0.1

Added note that Flush reads are not supported when burst mode for BAR2 is enabled.

Updated the list of configurations supported by the Avalon-MM and Avalon-MM with DMA variants.
2018.05.07 18.0

Changed all references to Cyclone® 10 to Cyclone® 10 GX.

2017.10.06 17.1 Made the following change to the user guide:
  • Added support for Cyclone® 10 GX devices.
  • Added optional parameter to invert the RX polarity.
  • Corrected Feature Comparison for all Hard IP for PCI Express IP Core table: The Avalon-MM DMA interface does not automatically handle out-of-order completions.
  • Added missing sequence of programming steps in DMA Descriptor Controller Registers.
  • Rebranded as Intel.
  • Corrected minor errors and typos.
2017.05.26 17.0 Made the following changes to the user guide:
  • Added note that starting with the Quartus® Prime Pro Edition Software, version 17.0, the QSF assignments in the following answer What assignments do I need for a PCIe Gen1, Gen2 or Gen3 design that targets an Arria® 10 ES2, ES3 or production device? are already included in the design.
2017.05.08 17.0

Made the following changes to the IP core:

  • Added option soft DFE Controller IP on the PHY tab of the parameter editor to improve BER margin. The default for this option is off because it is typically not required. Short reflective links may benefit from this soft DFE controller IP. This parameter is available only for Gen3 configurations.
Made the following changes to the user guide:
  • Updated PCI Express Gen3 Bank Usage Restrictions status. These restrictions affect all Aria 10 ES and production devices.
  • Corrected Table 2: Comparison for 128- and 256-Bit Avalon-MM with DMA Interface to the Application Layer. The 128-bit interface supports Gen3 x2 operation.
  • Clarified behavior of the Read Descriptor Controller and Write Descriptor Controller Avalon-MM Master interfaces. These Controllers send an MSI to the host upon completion of the last descriptor unless MSIs are disabled. By default, MSIs are enabled.
  • Corrected Comparison of Avalon-ST, Avalon-MM and Avalon-MM with DMA Interfaces table. Out-of-order Completions are not supported transparently for the Avalon-MM with DMA interface.
  • Corrected default values for the Uncorrectable Internal Error Mask Register and Correctable Internal Error Mask Register registers.
  • Added Understanding the Avalon-MM DMA Ports to the Getting Started with the Avalon-MM DMA Endpoint chapter.
  • Corrected minor errors and typos.
2017.03.15 16.1.1 Made the following changes:
  • Added statement that Arria® 10 devices do not support the Create timing and resource estimates for third-party EDA synthesis tools option on the Generate > Generate HDL menu.
  • Rebranded as Intel.
2016.10.28 16.1 Made the following to the IP core changes:
  • Increased the max DMA transfer to 1 MB for both the 128- and 256-bit interfaces.
  • Timing models are now final for most Arria® 10 device packages. Exceptions include some military and automotive speed grades with extended temperature ranges.

Made the following changes to the user guide:

  • Changed the recommended value of test_in[31:0] from 0xa8 to 0x188.
  • Removed recommendations about connecting pin_perst. These recommendations do not apply to Arria® 10 devices.
  • Corrected the number of tags supported in the Feature Comparison for all Hard IP for PCI Express IP Cores table.
  • Added PCIe bifurcation to the Feature Comparison for all Hard IP for PCI Express IP Cores table. PCI bifurcation is not supported.
  • Removed reference to a Linux software driver for the DMA modules which is not available.
  • Added section covering design example limitations.
  • Added -3 to recommended speed grades for the 125 MHz interface.
2016.05.02 16.0

Redesigned the 128-bit interface to the Application Layer resulting in consistently high throughput, for both on-chip and external memory.

In the Getting Started with the Avalon-MM DMA Endpoint chapter, changed the instructions to use specify the 10AX115S2F45I1SG device which is used on the Arria® 10 GX FPGA Development Kit - Production (not ES2) Edition.

Added support for Intel FPGA IP Evaluation Mode in the Quartus® Prime Pro Edition software.

Added simulation support for Gen3 PIPE mode using the ModelSim, VCS, and NCSim simulators.

Added automatic generation of basic Signal Tap Logic Analyzer files to facilitate debugging.

Revised discussion of the DMA Descriptor Controller in the Avalon-MM with DMA IP Core Architecture.

Revised Read DMA Example to reflect current maximum transfer size of 64 KB for 256-bit interface. The example now corresponds to an example design provided in the <install_dir>

Updated figures in Physical Layout of Hard IP in Arria® 10 Devices to include more detail about transceiver banks and channel restrictions.

Added Vendor Specific Extended Capability (VSEC) Revision and User Device or Board Type ID register from the Vendor Specific Extended Capability: to the VSEC tab of the component GUI.

Removed Arria® 10 PCI Express Quick Start Guide chapter. This chapter does not provide DMA functionality.

Corrected description of Write Descriptor Table Avalon-MM Slave Port.

Added Vendor Specific Extended Capability (VSEC) parameter descriptions which were missing from previous versions.

Added transceiver bank usage placement restrictions for Gen3 ES3 devices.

Removed support for -3 speed grade devices.

Added appendix listing previous versions of this user guide.

Corrected minor errors and typos.
2015.11.02 15.1 Made the following changes:
  • Added support for 256 tags to enhance throughput in high latency designs.
  • Added support for RX Completion buffer overflow monitoring.
  • To enhance performance and reduce internal buffering requirements, limited descriptor size to 8 KB.
  • Redesigned component GUI.
  • Added new Design Example tab that you can use to generate a design example you can download to the Altera Arria® 10 GX FPGA Development Kit.
  • Removed the parameter values High and Maximum from the RX buffer allocation parameter. These values are not supported for the Avalon-MM interface.
  • Enhanced the definition of npor.
  • Corrected resource utilization.
  • Clarified that conditions necessary before changing the base address for RC Read Status and Descriptor Base (Low) and RC Write Status and Descriptor Base (Low) registers.
  • Added an immediate write mode for single dword writes. The data is stored in the WR_RC_LOW_SRC_ADDR register. The new Immediate Write Mode bit of the DMA Descriptor controls this functionality.
  • Corrected TLP Support Comparison for all Hard IP for PCI Express IP Cores entries. Only Completions with and without data are supported for the Avalon-MM DMA interface. Message Requests with and without data are not supported for the Avalon-M interface.
  • Added optional Hard IP Status bus signals to the Avalon-MM DMA Bridge with Internal Descriptor Controller and Avalon-MM DMA Bridge with External Descriptor Controller figures.
2015.06.05 15.0 Added note inPhysical Layout of Hard IP in Arria® 10 Devices to explain Arria® 10 design constraint that requires that if the lower HIP on one side of the device is configured with a Gen3 x4 or Gen3 x8 IP core, and the upper HIP on the same side of the device is also configured with a Gen3 IP core, then the upper HIP must be configured with a x4 or x8 IP core.
2015.05.14 15.0 Made the following changes to the user guide:
  • Added Enable Hard IP Status Bus when using the AVMM interface parameter in Interface System Settings. This parameter is available in the IP core v15.0 and later.
2015.05.04 15.0
  • Added Enable Altera Debug Master Endpoint (ADME) parameter to support optional Native PHY register programming with the Altera System Console.
  • Added support for downstream burst read request for a payload of size up to 4 KB, if Enable burst capability for RXM BAR2 port is turned on in the Parameter Editor. Previous maximum downstream read request payload size was 512 bytes.
  • Corrected the allowed value of the Maximum payload size parameter for Avalon-MM DMA IP core variations, in Device Capabilities topic.
  • Corrected the supported variations to include Gen3 x2.
  • Removed the High and Maximum values for the RX Buffer credit allocation -performance for received requests parameter. These values are no longer valid settings. />.
  • Enhanced descriptions of channel placement, added fPLL placement for Gen1 and Gen2 data rates, and added master CGB location, in Physical Layout of Hard IP in Arria® 10 Devices. .
  • Reinstated Design Implementation chapter.
  • Added column for Avalon-ST Interface with SR-IOV variations in Feature Comparison for all Hard IP for PCI Express IP Cores table in the Features section. section.
  • Removed Migration and TLP Format appendices, and added new Frequently Asked Questionsappendix.
  • Updated information in SDC Timing Constraints section.
  • Removed list of static example designs from Design Examples. You can derive the list from the installation directory where example designs are available.
  • Fixed minor errors and typos.
2014.12.15 14.1 Made the following changes to the Arria® 10 user guide:

  • In the Getting Started chapter, corrected directory path for the simulation.

  • Added the fact that the RX Burst Master only support dword granularity.
  • Added definitions for test_in[2], test_in[6] and test_in[7].
  • Added instructions for Quartus II compilation.
2014.08.18 13.1 Arria® 10

Made the following changes to the Arria® 10 Avalon-MM DMA for PCI Express IP core:

  • Revised programming model for the Descriptor Controller.
  • Added simulation log file, altpcie_monitor_a10_dlhip_tlp_file_log.log, that is automatically generated in your simulation directory. To simulate in the Quartus II 14.0 software release, you must regenerate your IP core to create the supporting monitor file that generates altpcie_monitor_a10_dlhip_tlp_file_log.log. Refer to Understanding Simulation Dump File Generation for details.
  • Added support for either 128- or 256-bit interface to the Application Layer.
  • Added support for 64-bit addressing, making address translation unnecessary.
  • Removed Channel Placement for PCIe in Arria® 10 Devices. Please contact your Altera sales representative for PLL and channel usage.
  • Added support for optional bursting RX Master for BAR2.
  • Revised Read DMA Example and Software Program for Simultaneous Read and Write DMA to work with revised programming model for the Descriptor Controller.
  • Added the following optimizations for the Descriptor Controller:
    • Optimized performance for smaller payloads such as 64-byte Ethernet packets
    • Reduced overhead for host updates
    • Support for concurrent dynamic host updates and DMA operation
    • Support for choice to embed Descriptor Controller in the Avalon-MM bridge or instantiate separately
  • Added access to selected Configuration Space registers and link status registers through the optional Control Register Access (CRA) Avalon-MM slave port.
  • Added simulation support for Phase 2 and Phase 3 equalization when requested by third-party BFM for Gen3 variants.
  • Added multiple MSI/MSI-X support.

Made the following changes to the user guide:

  • Removed 125 MHz clock as optional refclk frequency in Arria® 10 devices. Arria® 10 devices support a 100 MHz reference clock as specified by the PCI Express Base Specification, Rev 3.0
  • Corrected values for Maximum payload size parameter. The sizes available are 128 or 256 bytes.
  • Enhanced definition of Device ID and Sub-system Vendor ID to say that these registers are only valid in the Type 0 (Endpoint) Configuration Space.
  • Removed 125 MHz clock as optional refclk frequency in Arria® 10 devices. Arria® 10 devices support an 100 MHz reference clock as specified by the PCI Express Base Specification, Rev 3.0.
  • Added Next Steps in Creating a Design for PCI Express to Datasheet chapter.
  • Removed the Transaction Layer Protocol Details chapter. This information only applies to the Avalon-ST interface.
  • Removed txdatavalid0 signal from the PIPE interface. This signal is not available.
  • Removed references to the MegaWizard® Plug-In Manager. In 14.0 the IP Parameter Editor Powered by Platform Designer has replaced the MegaWizard Plug-In Manager.
  • Added definitions for test_in[2], test_in[6] and test_in[7].
  • Corrected interface widths in the Performance and Resource Utilization Arria® 10 Avalon-MM DMA for PCI Express table in the Datasheet: Arria® 10 Avalon-MM DMA for PCIe chapter.
  • Removed discussion of pclk. This clock is not customer accessible in Arria® 10 devices.
  • Corrected Reset Controller in Arria® 10 Devices figure in Reset and Clocks chapter.
  • Corrected bit definitions for CvP Status register.
  • Removed PLL from channel placement figures.
  • Added fast passive parallel (FPP) to supported configuration schemes in CvP in Arria® 10 Devices figure.
  • Updated Power Supply Voltage Requirements table.
  • Corrected the name of the Descriptor Instructions bus. The letters DMA are now Ast. For example WrDMARXValid_i is now WrAstRXValid_i.
  • Added RD_CONTROL and WR_CONTROL register Done bit. When set, the Descriptor Controller writes this bit for each descriptor in the status table and sends a single MSI interrupt after the final descriptor completes.
  • Removed the following chapters that have minimal relevance to the Arria® 10 Avalon-MM DMA Interface IP Core. These chapters are available in the more comprehensive Avalon-ST versions :
    • Design Implementation
    • Optional Features
    • Debugging
    • Throughput Optimization
2013.12.02 13.1 Arria® 10

Initial release.

Level Two Title