Multi Channel DMA Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 683821
Date 10/06/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Before You Begin 2. Introduction 3. Functional Description 4. Interface Overview 5. Parameters (H-Tile) 6. Parameters (P-Tile) (F-Tile) (R-Tile) 7. Designing with the IP Core 8. Software Programming Model 9. Registers 10. Troubleshooting/Debugging 11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives 12. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express User Guide
1. Before You Begin x
1.1. Terms and Acronyms 1.2. Known Issues
2. Introduction x
2.1. Multi Channel DMA IP for PCI Express Features 2.2. Device Family Support 2.3. Recommended Speed Grades 2.4. Resource Utilization 2.5. Release Information
2.1. Multi Channel DMA IP for PCI Express Features x
2.1.1. Endpoint Mode 2.1.2. Root Port Mode
3. Functional Description x
3.1. Multi Channel DMA 3.2. Bursting Avalon-MM Master (BAM) 3.3. Bursting Avalon-MM Slave (BAS) 3.4. MSI Interrupt 3.5. Config Slave (CS) 3.6. Root Port Address Translation Table Enablement 3.7. Hard IP Reconfiguration Interface 3.8. Config TL Interface 3.9. Configuration Intercept Interface (EP Only) 3.10. Data Mover Only
3.1. Multi Channel DMA x
3.1.1. H2D Data Mover 3.1.2. D2H Data Mover 3.1.3. Descriptors 3.1.4. Avalon-MM PIO Master 3.1.5. Avalon-MM Write (H2D) and Read (D2H) Master 3.1.6. Avalon-ST Source (H2D) and Sink (D2H) 3.1.7. User MSI-X 3.1.8. User Functional Level Reset (FLR) 3.1.9. Control Registers
3.1.2. D2H Data Mover x
3.1.2.1. D2H Descriptor Fetch
3.1.3. Descriptors x
3.1.3.1. Support for Unaligned (or byte-aligned) Data Transfer 3.1.3.2. Metadata Support 3.1.3.3. MSI-X/Writeback
3.1.6. Avalon-ST Source (H2D) and Sink (D2H) x
3.1.6.1. Avalon-ST 1-Port Mode 3.1.6.2. Packet (File) Boundary 3.1.6.3. Metadata
3.4. MSI Interrupt x
3.4.1. Endpoint MSI Support through BAS 3.4.2. MSI Interrupt Controller
3.5. Config Slave (CS) x
3.5.1. 29 bit AVMM address format 3.5.2. 14 bit AVMM address format 3.5.3. Configuration Access Mechanism
3.10. Data Mover Only x
3.10.1. H2D Data Mover 3.10.2. D2H Data Mover 3.10.3. Application Specific Bits
3.10.1. H2D Data Mover x
3.10.1.1. H2D Descriptor Format (h2ddm_desc) 3.10.1.2. H2D Descriptor Completion Packet Format (h2ddm_desc_cmpl) 3.10.1.3. H2D Descriptor Status (h2ddm_desc_status)
3.10.2. D2H Data Mover x
3.10.2.1. D2H Descriptor Format (d2hdm_desc) 3.10.2.2. D2H Descriptor Status (d2hdm_desc_status)
4. Interface Overview x
4.1. Port List 4.2. Clocks 4.3. Resets 4.4. Multi Channel DMA 4.5. Bursting Avalon-MM Master (BAM) Interface 4.6. Bursting Avalon-MM Slave (BAS) Interface 4.7. MSI Interface 4.8. Config Slave Interface (RP only) 4.9. Hard IP Reconfiguration Interface 4.10. Config TL Interface 4.11. Configuration Intercept Interface (EP Only) 4.12. Data Mover Interface 4.13. Hard IP Status Interface
4.1. Port List x
4.1.1. Port List (H-Tile) 4.1.2. Port List (P-Tile) (F-Tile) (R-Tile)
4.2. Clocks x
4.2.1. F-Tile System PLL Reference Clock Requirements
4.4. Multi Channel DMA x
4.4.1. Avalon-MM PIO Master 4.4.2. Avalon-MM Write Master (H2D) 4.4.3. Avalon-MM Read Master (D2H) 4.4.4. Avalon-ST Source (H2D) 4.4.5. Avalon-ST Sink (D2H) 4.4.6. User Event MSI-X Interface 4.4.7. User Functional Level Reset (FLR) Interface
4.12. Data Mover Interface x
4.12.1. H2D Data Mover Interface 4.12.2. D2H Data Mover Interface
5. Parameters (H-Tile) x
5.1. IP Settings 5.2. Example Designs
5.1. IP Settings x
5.1.1. System Settings 5.1.2. MCDMA Settings 5.1.3. Device Identification Registers 5.1.4. Multifunction and SR-IOV System Settings Parameters [Endpoint Mode] 5.1.5. Configuration, Debug and Extension Options 5.1.6. PHY Characteristics 5.1.7. PCI Express / PCI Capabilities Parameters
5.1.7. PCI Express / PCI Capabilities Parameters x
5.1.7.1. Device 5.1.7.2. Link 5.1.7.3. MSI-X 5.1.7.4. Power Management 5.1.7.5. Vendor Specific Extended Capability (VSEC)
6. Parameters (P-Tile) (F-Tile) (R-Tile) x
6.1. Top-Level Settings 6.2. PCIe0 Settings 6.3. Example Designs 6.4. Analog Parameters (F-Tile MCDMA IP Only) 6.5. PCIe1 Settings
6.2. PCIe0 Settings x
6.2.1. Base Address Register 6.2.2. PCIe0 Configuration, Debug and Extension Options 6.2.3. PCIe0 Device Identification Registers 6.2.4. PCIe0 PCI Express / PCI Capabilities 6.2.5. MCDMA Settings
6.2.3. PCIe0 Device Identification Registers x
6.2.3.1. PCIe0 PF0 IDs 6.2.3.2. PCIe0 PF0 VF IDs
6.2.4. PCIe0 PCI Express / PCI Capabilities x
6.2.4.1. PCIe0 Device 6.2.4.2. PCIe0 Link 6.2.4.3. PCIe0 MSI-X 6.2.4.4. PCIe0 DEV SER 6.2.4.5. PCIe0 PRS 6.2.4.6. PCIe0 PTM 6.2.4.7. PCIe0 ACS Capabilities 6.2.4.8. PCIe0 ATS 6.2.4.9. PCIe0 TPH
6.2.4.7. PCIe0 ACS Capabilities x
6.2.4.7.1. PCIe0 ACS for Physical Functions 6.2.4.7.2. PCIe0 ACS for Virtual Functions
6.2.4.8. PCIe0 ATS x
6.2.4.8.1. PCIe0 ATS for Physical Functions 6.2.4.8.2. PCIe0 ATS for Virtual Functions
6.2.4.9. PCIe0 TPH x
6.2.4.9.1. PCIe0 TPH for Physical Functions 6.2.4.9.2. PCIe0 TPH for Virtual Functions
6.5. PCIe1 Settings x
6.5.1. PCIe1 Configuration, Debug and Extension Options
7. Designing with the IP Core x
7.1. Generating the IP Core 7.2. Simulating the IP Core 7.3. IP Core Generation Output - Intel® Quartus® Prime Pro Edition 7.4. Systems Integration and Implementation
7.4. Systems Integration and Implementation x
7.4.1. Required Supporting IP
8. Software Programming Model x
8.1. Multi Channel DMA IP Custom Driver 8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver 8.3. Multi Channel DMA IP Kernel Mode Network Device Driver
8.1. Multi Channel DMA IP Custom Driver x
8.1.1. Architecture 8.1.2. libmqdma library details 8.1.3. Application 8.1.4. Software Flow 8.1.5. API Flow 8.1.6. libmqdma Library API List 8.1.7. Request Structures
8.1.2. libmqdma library details x
8.1.2.1. Channel Initialization 8.1.2.2. Descriptor Memory Management 8.1.2.3. Descriptor Completion Status Update
8.1.5. API Flow x
8.1.5.1. Single Descriptor Load and Submit 8.1.5.2. Multiple Descriptor Load and Submit
8.1.6. libmqdma Library API List x
8.1.6.1. ifc_api_start 8.1.6.2. ifc_mcdma_port_by_name 8.1.6.3. ifc_qdma_device_get 8.1.6.4. ifc_num_channels_get 8.1.6.5. ifc_qdma_channel_get 8.1.6.6. ifc_qdma_acquire_channels 8.1.6.7. ifc_qdma_release_all_channels 8.1.6.8. ifc_qdma_device_put 8.1.6.9. ifc_qdma_channel_put 8.1.6.10. ifc_qdma_completion_poll 8.1.6.11. ifc_qdma_request_start 8.1.6.12. ifc_qdma_request_prepare 8.1.6.13. ifc_qdma_descq_queue_batch_load 8.1.6.14. ifc_qdma_request_submit 8.1.6.15. ifc_qdma_pio_read32 8.1.6.16. ifc_qdma_pio_write32 8.1.6.17. ifc_qdma_pio_read64 8.1.6.18. ifc_qdma_pio_write64 8.1.6.19. ifc_qdma_pio_read128 8.1.6.20. ifc_qdma_pio_write128 8.1.6.21. ifc_qdma_pio_read256 8.1.6.22. ifc_qdma_pio_write256 8.1.6.23. ifc_request_malloc 8.1.6.24. ifc_request_free 8.1.6.25. ifc_app_stop 8.1.6.26. ifc_qdma_poll_init 8.1.6.27. ifc_qdma_poll_add 8.1.6.28. ifc_qdma_poll_wait 8.1.6.29. ifc_mcdma_port_by_name
8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver x
8.2.1. Architecture 8.2.2. MCDMA Poll Mode Driver 8.2.3. DPDK based application 8.2.4. Request Structures 8.2.5. Software Flow 8.2.6. API Flow 8.2.7. API List
8.2.2. MCDMA Poll Mode Driver x
8.2.2.1. Completion Status Update 8.2.2.2. Metadata Support 8.2.2.3. User MSI-X
8.3. Multi Channel DMA IP Kernel Mode Network Device Driver x
8.3.1. Architecture 8.3.2. Driver Information 8.3.3. Software Flow
8.3.2. Driver Information x
8.3.2.1. Device Management 8.3.2.2. Channel Management 8.3.2.3. Descriptor Memory Management 8.3.2.4. Completions Management 8.3.2.5. ethtool support 8.3.2.6. debugfs support 8.3.2.7. SRIOV Support
8.3.3. Software Flow x
8.3.3.1. Host to Device Flow 8.3.3.2. Device to Host Flow
9. Registers x
9.1. Queue Control (QCSR) 9.2. MSI-X Memory Space 9.3. Control Register (GCSR)
10. Troubleshooting/Debugging x
10.1. Debug Toolkit
10.1. Debug Toolkit x
10.1.1. Overview 10.1.2. Enabling the Debug Toolkit 10.1.3. Launching the Debug Toolkit 10.1.4. Using the Debug Toolkit
10.1.4. Using the Debug Toolkit x
10.1.4.1. Main View 10.1.4.2. Toolkit Parameters 10.1.4.3. Channel Parameters 10.1.4.4. Eye Viewer 10.1.4.5. Link Inspector
10.1.4.4. Eye Viewer x
10.1.4.4.1. Running Eye Viewer in the P-Tile Debug Toolkit 10.1.4.4.2. Running Eye Viewer in the F-Tile Debug Toolkit
1. Before You Begin
2. Introduction
3. Functional Description
4. Interface Overview
5. Parameters (H-Tile)
6. Parameters (P-Tile) (F-Tile) (R-Tile)
7. Designing with the IP Core
8. Software Programming Model
9. Registers
10. Troubleshooting/Debugging
11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives
12. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express User Guide

6.1. Top-Level Settings

Figure 27. Multi Channel DMA IP for PCI Express Parameter Editor
Table 70.  Top-Level Settings

Parameter

Value

Default Value

Description

Hard IP mode (P-Tile)

Gen4 x16, Interface – 512 bit

Gen3 x16, Interface – 512 bit

Gen4 2x8, Interface - 256 bit

Gen3 2x8, Interface - 256 bit

Gen4 1x8, Interface – 256 bit

Gen3 1x8, Interface – 256 bit

Gen4 4x4, Interface - 128 bit

Gen3 4x4, Interface - 128 bit

Gen4x16, Interface – 512 bit

Selects the following elements:
  • Lane data rate: Gen3 and Gen4 are supported.
  • Lane width: x16 supports both Root Port and Endpoint modes. 1 x8, 2 x8 support only Endpoint mode. 4 x4 mode only supports Root Port mode.
Hard IP mode (F-Tile)

Gen4 x16, Interface – 512 bit

Gen3 x16, Interface – 512 bit

Gen4 2x8, Interface - 256 bit

Gen3 2x8, Interface - 256 bit

Gen4 1x8, Interface – 256 bit

Gen3 1x8, Interface – 256 bit

Gen4 1x4, Interface - 128 bit

Gen3 1x4, Interface - 128 bit

Gen4 2x4, Interface - 128 bit

Gen3 2x4, Interface - 128 bit

Gen4 4x4, Interface - 128 bit

Gen3 4x4, Interface - 128 bit

Gen4x16, Interface – 512 bit

Selects the following elements:
  • Lane data rate: Gen3 and Gen4 are supported.
  • Lane width: x16 and 1 x8 support both Root Port and Endpoint mode. 2 x8 and 1 x4 support only Endpoint mode. 2 x4 and 4 x4 support only Root Port mode.
Hard IP mode (R-Tile)

Gen4 x16, Interface – 512 bit

Gen3 x16, Interface – 512 bit

Gen5 2x8, Interface – 512 bit

Gen4 2x8, Interface - 512 bit

Gen3 2x8, Interface - 512 bit

Gen4 2x8, Interface – 256 bit

Gen3 2x8, Interface – 256 bit

Gen5 4x4, Interface - 256 bit

Gen4 4x4, Interface - 256 bit

Gen3 4x4, Interface - 256 bit

Gen4 4x4, Interface - 128 bit

Gen3 4x4, Interface - 128 bit

Gen5 2x8, Interface – 512 bit
Selects the following elements:
  • Link data rate: Gen5, Gen4 and Gen3 are supported.
  • Link width: x16, x8 and x4 modes supported for both Root Port and Endpoint
DK-DEV-AGI0227RES R-Tile A0 revision only supports:
  • Gen5 2x8, Interface - 512 bit
  • Gen4 2x8, Interface - 512 bit
  • Gen3 2x8, Interface - 512 bit
Number of PCIe 1 / 2 1

Display total number of MCDMA IP cores in x8 mode.
Port Mode

Native Endpoint

Root Port

Native Endpoint

Specifies the port type.

Root Port mode: x16,1x8 (F-Tile only), 2x8 (R-Tile only), 2x4 (F-Tile only), 4x4

Endpoint mode: x16, 1x8, 2x8, 1x4 (F-Tile only), 4x4 (R-Tile only)

Note: 1x8 is not supported for R-Tile
Note: In R-Tile 4x4, Port mode has several options EP/EP/EP/EP or EP/EP/RP/RP or EP/RP/RP/RP or RP/RP/RP/RP.

Enable Ptile Debug Toolkit (P-Tile)

Enable Debug Toolkit (F-Tile and R-Tile)

On / Off

Off

Enable the Debug Toolkit for JTAG-based System Console debug access.

Enable PHY Reconfiguration

On / Off

Off

When on, creates an Avalon-MM slave interface that software can drive to update Transceiver reconfiguration registers

Enable the transceiver PMA registers access through a dedicated an Avalon-MM slave interface.

Note: In F-Tile, this option has renamed as Enable PMA registers access
Note: This parameter is not supported for R-Tile

PLD Clock Frequency (P-Tile and F-Tile)

500 MHz

450 MHz

400 MHz

350 MHz

250 MHz

225 MHz

200 MHz

175 MHz

350 MHz (for Gen4 modes)

250 MHz (for Gen3 modes)

Select the frequency of the Application clock. The options available vary depending on the setting of the Hard IP Mode parameter.

For Gen4 modes, the available clock frequencies are 500 MHz / 450 MHz / 400 MHz / 350 MHz / 250 MHz / 225 MHz / 200 MHz / 175 MHz (for Intel Agilex® 7) and 400 MHz / 350 MHz / 200 MHz /175 MHz (for Intel Stratix 10 DX).

For Gen3 modes, the available clock frequency is 250 MHz (for Intel Agilex® 7 and Intel Stratix 10 DX).

PLD Clock Frequency (R-Tile)

500 MHz

475 MHz

450 MHz

425 MHz

400 MHz

350 MHz

275 MHz

250 MHz

500 MHz (for Gen5 mode)

500 MHz or 300 MHz (for Gen4 mode)

250 MHz (for Gen3 mode)

Selects the frequency of the Application clock. The options available vary depending on the setting of the Hard IP Mode parameter.

For Gen5 modes, the available clock frequencies are 500 MHz / 475 MHz / 450 MHz / 425 MHz / 400 MHz

For Gen4 modes, the available clock frequencies are 500 MHz / 475 MHz / 450 MHz / 425 MHz / 400 MHz / 300 MHz / 275 MHz / 250 MHz

For Gen3 modes, the available clock frequency are 300 MHz / 275 MHz / 250 MHz
Enable SRIS Mode

On / Off

Off

Enable the Separate Reference Clock with Independent Spread Spectrum Clocking (SRIS) feature.

When you enable this option, the Slot clock configuration option under the PCIe Settings → PCIe PCI Express/PCI Capabilities → PCIe Link tab will be automatically disabled.

P-Tile Sim Mode

On / Off

Off

Enabling this parameter reduces the simulation time of Hot Reset tests by 5 ms.

Note: Do not enable this option if you need to run synthesis.
Note: This parameter is not supported for R-Tile and F-Tile.

Enable PIPE Mode Simulation

On / Off

Off

When you set this parameter, the PIPE interface is exposed which can be used to improve the simulation time.

Note: This parameter is only supported for R-Tile MCDMA IP.
Note: This parameter is present in F-Tile MCDMA IP. However, F-Tile MCDMA Design Examples do not support PIPE mode Simulation.
Note: When running simulations with the PIPE interface, the following macro is required: "+define+RTILE_PIPE_MODE"

For more information regarding PIPE Mode Simulation feature and its usage, refer to R-Tile Avalon Streaming Intel FPGA IP for PCI Express User Guide.

Enable Independent Perst (P-Tile and F-Tile)

Enable Independent GPIO Perst (R-Tile)

On / Off Off

Enable the reset of PCS and Controller in User Mode for Endpoint 2x8 mode.

When this parameter is On, new signals p<n>_cold_perst_n_i and p<n>_warm_perst_n_i are exported to the user application for P/R-Tiles. In case of F-Tile, i_gpio_perst#_n is exported to the user application..

When this parameter is Off (default), the IP internally ties off these signals instead of exporting them.

Note: This parameter is required for the independent reset feature, which is only supported in the x8x8 Endpoint/Endpoint mode. In F-Tile, the Hard IP Reconfiguration Interface must be enabled and p0_hip_reconfig_clk port must be connected to a clock source when it is using this reset signal or Enable Independent Perst option is turned on.
Note: For more information regarding the independent resets feature and its usage, refer to
Enable CVP (Intel VSEC) On / Off Off

Enable support for CVP flow for single tile only

Refer to Intel Agilex® 7 Device Configuration via Protocol (CvP) Implementation User Guide for more information

Slow Clock Divider

2 & 4

4

Allows you to set the slow_clk to be divided by 2 or 4 from the coreclkout_hip.

Note: This parameter is supported for R-Tile only.
Level Two Title