Multi Channel DMA Intel® FPGA IP for PCI Express User Guide

Download PDF
ID 683821
Date 11/01/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Before You Begin 2. Introduction 3. Functional Description 4. Interface Overview 5. Parameters (H-Tile) 6. Parameters (P-Tile and F-Tile) 7. Designing with the IP Core 8. Software Programming Model 9. Registers 10. Troubleshooting/Debugging 11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives 12. Revision History for Multi Channel DMA Intel FPGA IP for PCI Express User Guide
1. Before You Begin x
1.1. Terms and Acronyms 1.2. Known Issues
2. Introduction x
2.1. Multi Channel DMA IP for PCI Express Features 2.2. Device Family Support 2.3. Recommended Speed Grades 2.4. Resource Utilization 2.5. Release Information
2.1. Multi Channel DMA IP for PCI Express Features x
2.1.1. Endpoint Mode 2.1.2. Root Port Mode
3. Functional Description x
3.1. Multi Channel DMA 3.2. Bursting Avalon-MM Master (BAM) 3.3. Bursting Avalon-MM Slave (BAS) 3.4. MSI Interrupt 3.5. Config Slave (CS) 3.6. Root Port Address Translation Table Enablement 3.7. Hard IP Reconfiguration Interface 3.8. Config TL Interface 3.9. Configuration Intercept Interface (EP Only) 3.10. Data Mover Only
3.1. Multi Channel DMA x
3.1.1. H2D Data Mover 3.1.2. D2H Data Mover 3.1.3. Descriptors 3.1.4. Avalon-MM PIO Master 3.1.5. Avalon-MM Write (H2D) and Read (D2H) Master 3.1.6. Avalon-ST Source (H2D) and Sink (D2H) 3.1.7. User MSI-X 3.1.8. User Functional Level Reset (FLR) 3.1.9. Control Registers
3.1.2. D2H Data Mover x
3.1.2.1. D2H Descriptor Fetch
3.1.3. Descriptors x
3.1.3.1. Support for Unaligned (or byte-aligned) Data Transfer 3.1.3.2. Metadata Support 3.1.3.3. MSI-X/Writeback
3.1.6. Avalon-ST Source (H2D) and Sink (D2H) x
3.1.6.1. Avalon-ST 1-Port Mode 3.1.6.2. Packet (File) Boundary 3.1.6.3. Metadata
3.4. MSI Interrupt x
3.4.1. Endpoint MSI Support through BAS 3.4.2. MSI Interrupt Controller
3.5. Config Slave (CS) x
3.5.1. 29 bit AVMM address format 3.5.2. 14 bit AVMM address format 3.5.3. Configuration Access Mechanism
3.10. Data Mover Only x
3.10.1. H2D Data Mover 3.10.2. D2H Data Mover 3.10.3. Application Specific Bits
3.10.1. H2D Data Mover x
3.10.1.1. H2D Descriptor Format (h2ddm_desc) 3.10.1.2. H2D Descriptor Completion Packet Format (h2ddm_desc_cmpl) 3.10.1.3. H2D Descriptor Status (h2ddm_desc_status)
3.10.2. D2H Data Mover x
3.10.2.1. D2H Descriptor Format (d2hdm_desc) 3.10.2.2. D2H Descriptor Status (d2hdm_desc_status)
4. Interface Overview x
4.1. Port List 4.2. Clocks 4.3. Resets 4.4. Multi Channel DMA 4.5. Bursting Avalon-MM Master (BAM) Interface 4.6. Bursting Avalon-MM Slave (BAS) Interface 4.7. MSI Interface (EP) 4.8. Config Slave Interface (RP only) 4.9. Hard IP Reconfiguration Interface 4.10. Config TL Interface 4.11. Configuration Intercept Interface (EP Only) 4.12. Data Mover Interface 4.13. Hard IP Status Interface
4.1. Port List x
4.1.1. Port List (H-Tile) 4.1.2. Port List (P-Tile and F-Tile)
4.4. Multi Channel DMA x
4.4.1. Avalon-MM PIO Master 4.4.2. Avalon-MM Write Master (H2D) 4.4.3. Avalon-MM Read Master (D2H) 4.4.4. Avalon-ST Source (H2D) 4.4.5. Avalon-ST Sink (D2H) 4.4.6. User Event MSI-X Interface 4.4.7. User Functional Level Reset (FLR) Interface
4.12. Data Mover Interface x
4.12.1. H2D Data Mover Interface 4.12.2. D2H Data Mover Interface
5. Parameters (H-Tile) x
5.1. IP Settings 5.2. Example Designs
5.1. IP Settings x
5.1.1. System Settings 5.1.2. MCDMA Settings 5.1.3. Device Identification Registers 5.1.4. Multifunction and SR-IOV System Settings Parameters [Endpoint Mode] 5.1.5. Configuration, Debug and Extension Options 5.1.6. PHY Characteristics 5.1.7. PCI Express / PCI Capabilities Parameters
5.1.7. PCI Express / PCI Capabilities Parameters x
5.1.7.1. Device 5.1.7.2. Link 5.1.7.3. MSI-X 5.1.7.4. Power Management 5.1.7.5. Vendor Specific Extended Capability (VSEC)
6. Parameters (P-Tile and F-Tile) x
6.1. Top-Level Settings 6.2. PCIe0 Settings 6.3. PCIe1 Settings 6.4. Example Designs
6.2. PCIe0 Settings x
6.2.1. Base Address Register 6.2.2. PCIe0 Configuration, Debug and Extension Options 6.2.3. PCIe0 Device Identification Registers 6.2.4. PCIe0 PCI Express / PCI Capabilities Parameters 6.2.5. MCDMA Settings
6.2.3. PCIe0 Device Identification Registers x
6.2.3.1. PCIe0 PF0 IDs 6.2.3.2. PCIe0 PF0 VF IDs
6.2.4. PCIe0 PCI Express / PCI Capabilities Parameters x
6.2.4.1. PCIe0 Device 6.2.4.2. PCIe0 Link 6.2.4.3. PCIe0 MSI-X 6.2.4.4. PCIe0 DEV SER 6.2.4.5. PCIe0 PRS 6.2.4.6. PCIe0 ACS Capabilities 6.2.4.7. PCIe0 ATS 6.2.4.8. PCIe0 TPH
6.2.4.6. PCIe0 ACS Capabilities x
6.2.4.6.1. PCIe0 ACS for Physical Functions 6.2.4.6.2. PCIe0 ACS for Virtual Functions
6.2.4.7. PCIe0 ATS x
6.2.4.7.1. PCIe0 ATS for Physical Functions 6.2.4.7.2. PCIe0 ATS for Virtual Functions
6.2.4.8. PCIe0 TPH x
6.2.4.8.1. PCIe0 TPH for Physical Functions 6.2.4.8.2. PCIe0 TPH for Virtual Functions
6.3. PCIe1 Settings x
6.3.1. PCIe1 Configuration, Debug and Extension Options
7. Designing with the IP Core x
7.1. Generating the IP Core 7.2. Simulating the IP Core 7.3. IP Core Generation Output - Intel® Quartus® Prime Pro Edition 7.4. Systems Integration and Implementation
7.4. Systems Integration and Implementation x
7.4.1. Required Supporting IP
8. Software Programming Model x
8.1. Multi Channel DMA IP Custom Driver 8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver 8.3. Multi Channel DMA IP Kernel Mode Character Device Driver 8.4. Multi Channel DMA IP Kernel Mode Network Device Driver
8.1. Multi Channel DMA IP Custom Driver x
8.1.1. Architecture 8.1.2. libmqdma library details 8.1.3. Application 8.1.4. Software Flow 8.1.5. API Flow 8.1.6. libmqdma Library API List 8.1.7. Request Structures
8.1.2. libmqdma library details x
8.1.2.1. Channel Initialization 8.1.2.2. Descriptor Memory Management 8.1.2.3. Descriptor Completion Status Update
8.1.5. API Flow x
8.1.5.1. Single Descriptor Load and Submit 8.1.5.2. Multiple Descriptor Load and Submit
8.1.6. libmqdma Library API List x
8.1.6.1. ifc_api_start 8.1.6.2. ifc_mcdma_port_by_name 8.1.6.3. ifc_qdma_device_get 8.1.6.4. ifc_num_channels_get 8.1.6.5. ifc_qdma_channel_get 8.1.6.6. ifc_qdma_acquire_channels 8.1.6.7. ifc_qdma_release_all_channels 8.1.6.8. ifc_qdma_device_put 8.1.6.9. ifc_qdma_channel_put 8.1.6.10. ifc_qdma_completion_poll 8.1.6.11. ifc_qdma_request_start 8.1.6.12. ifc_qdma_request_prepare 8.1.6.13. ifc_qdma_descq_queue_batch_load 8.1.6.14. ifc_qdma_request_submit 8.1.6.15. ifc_qdma_pio_read32 8.1.6.16. ifc_qdma_pio_write32 8.1.6.17. ifc_qdma_pio_read64 8.1.6.18. ifc_qdma_pio_write64 8.1.6.19. ifc_qdma_pio_read128 8.1.6.20. ifc_qdma_pio_write128 8.1.6.21. ifc_qdma_pio_read256 8.1.6.22. ifc_qdma_pio_write256 8.1.6.23. ifc_request_malloc 8.1.6.24. ifc_request_free 8.1.6.25. ifc_app_stop 8.1.6.26. ifc_qdma_poll_init 8.1.6.27. ifc_qdma_poll_add 8.1.6.28. ifc_qdma_poll_wait 8.1.6.29. ifc_mcdma_port_by_name
8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver x
8.2.1. Architecture 8.2.2. MCDMA Poll Mode Driver 8.2.3. DPDK based application 8.2.4. Request Structures 8.2.5. Software Flow 8.2.6. API Flow 8.2.7. API List
8.2.2. MCDMA Poll Mode Driver x
8.2.2.1. Completion Status Update 8.2.2.2. Metadata Support 8.2.2.3. User MSI-X
8.3. Multi Channel DMA IP Kernel Mode Character Device Driver x
8.3.1. Architecture 8.3.2. libmcmem Library Information 8.3.3. Kernel Driver Information 8.3.4. Control Message Structure 8.3.5. Software Flow 8.3.6. API Flow 8.3.7. API List
8.3.3. Kernel Driver Information x
8.3.3.1. Device Management 8.3.3.2. Channel Management 8.3.3.3. Completions Management
8.3.3.3. Completions Management x
8.3.3.3.1. Interrupt Mode 8.3.3.3.2. Poll Mode
8.4. Multi Channel DMA IP Kernel Mode Network Device Driver x
8.4.1. Architecture 8.4.2. Driver Information 8.4.3. Software Flow
8.4.2. Driver Information x
8.4.2.1. Device Management 8.4.2.2. Channel Management 8.4.2.3. Descriptor Memory Management 8.4.2.4. Completions Management 8.4.2.5. ethtool support 8.4.2.6. debugfs support 8.4.2.7. SRIOV Support
8.4.3. Software Flow x
8.4.3.1. Host to Device Flow 8.4.3.2. Device to Host Flow
9. Registers x
9.1. Queue Control (QCSR) 9.2. MSI-X Memory Space 9.3. Control Register (GCSR)
10. Troubleshooting/Debugging x
10.1. Debug Toolkit
10.1. Debug Toolkit x
10.1.1. Overview 10.1.2. Enabling the Debug Toolkit 10.1.3. Launching the Debug Toolkit 10.1.4. Using the Debug Toolkit
10.1.4. Using the Debug Toolkit x
10.1.4.1. Main View 10.1.4.2. Toolkit Parameters 10.1.4.3. Channel Parameters General PHY TX Path RX Path 10.1.4.4. Eye Viewer 10.1.4.5. Link Inspector
1. Before You Begin
2. Introduction
3. Functional Description
4. Interface Overview
5. Parameters (H-Tile)
6. Parameters (P-Tile and F-Tile)
7. Designing with the IP Core
8. Software Programming Model
9. Registers
10. Troubleshooting/Debugging
11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives
12. Revision History for Multi Channel DMA Intel FPGA IP for PCI Express User Guide

10.1.4.3. Channel Parameters

The channel parameters window allows you to read the transmitter and receiver settings for a given channel. It has the following 3 sub-windows. Use the Lane Refresh button to read the status of the General PHY, TX Path, and RX Path sub-windows for each channel.

Note: To refresh channel parameters for more than one lanes simultaneously, select the lanes under the Collection tab, right click and select Refresh.

General PHY

This tab shows the reset status of the PHY. In the F-Tile debug toolkit, the PHY Reset is not available. The reset status is indicated by PIPE PhyStatus under Tile Information tab.

Figure 67. Channel Parameters Tab
Table 168.  General PHY Settings
  Parameters Values Descriptions
PHY Status PHY reset (For P-Tile debug toolkit only) Normal, Reset

Indicates the PHY is in reset mode.

Normal: PHY is out of reset.

Reset: PHY is in reset.
Event Counter 2 to clear the counter values. Elastic buffer overflow Hex value Indicates elastic buffer overflow errors.
Elastic buffer underflow Hex value Indicates elastic buffer underflow errors.
Decode error Hex value Indicates decode errors.
Running disparity error Hex value Indicates running disparity errors.
SYNC header error Hex value Indicates SYNC header errors.
RX valid deassertion without EIEOS Hex value Indicates errors when RX valid deassertion occurs without EIEOS.

TX Path

This tab allows you to monitor the transmitter settings for the channel selected.

Table 169.  Transmitter Settings
  Parameters Values Descriptions
TX Status TX Reset (For P-Tile debug toolkit only) Normal, Reset

Indicates if TX (TX datapath, TX settings) is in reset or normal operating mode.

Normal: TX is in normal operating mode.

Reset: TX is in reset.
TX Electrical Idle True, False

Indicates if TX is in electrical idle.

True: indicates TX is in electrical idle.

False: indicates TX is out of electrical idle.

TX PLL TX PLL lock Green, Red

Indicates if TX PLL is locked. This is dependent on the PLL selected as indicated by TX PLL select.

In P-Tile, there is one set of PLLs per Quad. The TX path of each channel reads out the PLL status corresponding to that Quad.

In F-tile, there is one PLL per channel. The TX path of the channel reads out the PLL status corresponding to that channel.

  • TX path for Ch0 to 3: Status of PLLs in Quad0
  • TX path for Ch4 to 7: Status of PLLs in Quad1
  • TX path for Ch8 to 11: Status of PLLs in Quad2
  • TX path for Ch12 to 15: Status of PLLs in Quad3

Green: TX PLL is locked.

Red: TX PLL is not locked.
TX VOD (For P-Tile debug toolkit only) Iboost level

Gen3: 15

Gen4: 15

Indicates the transmitter current boost level when the TX amplitude boost mode is enabled.

Vboost en

Gen3: Enable

Gen4: Enable

Indicates if the TX swing boost level is enabled.

Enable: TX swing boost is enabled.

Disable: TX swing boost is disabled.
Vboost level

Gen3: 5

Gen4: 5

 Indicates the TX Vboost level.
TX Equalization TX Equalization Status Not attempted, Completed, Unsuccessful Indicates transmitter equalization status. The TX local and remote parameters are valid only when the value of Equalization status is returned as completed, indicating equalization has completed successfully.
TX Local Preset P0 to P10 Indicates the transmitter driver preset value as requested by the link partner during the Equalization phase of link training. If the preset is not one of these values, then no value is shown.
Local Pre-shoot coefficient

Depends on the coefficient requested by the link partner.

Indicates the transmitter driver output pre-emphasis (pre-cursor coefficient value).
Local main coefficient

Depends on the coefficient requested by the link partner.

Indicates the transmitter driver output pre-emphasis (main cursor coefficient value).
Local post coefficient

Depends on the coefficient requested by the link partner.

Indicates the transmitter driver output pre-emphasis (post-cursor coefficient value).
Remote Pre-shoot coefficient (†) Depends on the transmitter driver output of the link partner.

Indicates link partner's transmitter driver's output pre-cursor coefficient value, as received by P-Tile or F-Tile during the Equalization phase of link training. When P-Tile or F-Tile is configured in Endpoint mode, this value corresponds to the coefficient received during Phase 2 of Equalization.

Remote main coefficient (†) Depends on the transmitter driver output of the link partner.

Indicates link partner's transmitter driver's output main cursor coefficient value, as received by P-Tile or F-Tile during the Equalization phase of link training. When P-Tile or F-Tile is configured in Endpoint mode, this value corresponds to the coefficient received during Phase 2 of Equalization.

Remote post coefficient (†) Depends on the transmitter driver output of the link partner. Indicates the link partner's transmitter driver's output post-cursor coefficient value, as received by P-Tile or F-Tile during the Equalization phase of link training. When P-Tile or F-Tile is configured in Endpoint mode, this value corresponds to the coefficient received during Phase 2 of Equalization.
Remote full swing (fs) (†) Depends on the device capability of the link partner. Indicates the full swing value used by the link partner during the Equalization phase of link training.
Remote low frequency (lf) (†) Depends on the device capability of the link partner.

Indicates the low frequency value used by the link partner during the Equalization phase of link training.

Note: (†) Refer to the following sections of the PCI Express* Base Specification Revision 4.0: 4.2.3 Link Equalization Procedure for 8.0 GT/s and Higher Data Rates and 8.3.3 Tx Voltage Parameters.
Figure 68. Example of Transmitter Settings

RX Path

This tab allows you to monitor and control the receiver settings for the channel selected.
Table 170.  Receiver Settings
  Parameters Values Descriptions
RX Status RX Reset (For P-Tile debug toolkit only) Normal, Reset

Indicates if RX (RX datapath, RX settings) is in reset or normal operating mode.

Normal: RX is in normal operating mode.

Reset: RX is in reset.
RX Polarity No polarity inversion, Polarity inversion

Indicates RX polarity inversion for the selected lane.

No polarity inversion: no polarity inversion on RX.

Polarity inversion: polarity inversion on RX.
RX Electrical Idle True, False

Indicates if RX is in electrical idle or not.

True: RX is in electrical idle.

False: RX is out of electrical idle.
Receiver Detected Green, Grey

Green: Far end receiver is detected.

Grey: Far end receiver is not detected.
RX CDR CDR Lock Green, Red

Indicates the CDR lock state.

Green: CDR is locked.

Red: CDR is not locked.
CDR Mode Locked to Reference (LTR), Locked to Data (LTD)

Indicates the CDR lock mode.

LTR: CDR is locked to reference clock.

LTD: CDR is locked to data.
RX Equalization RX ATT (For P-Tile debug toolkit only)

Gen3: 0

Gen4: 0

Indicates the RX equalization attenuation level.
RX CTLE Boost (For P-Tile debug toolkit only)

Gen3: 12

Gen4: 16

Indicates the RX CTLE boost value.
RX CTLE Pole (For P-Tile debug toolkit only)

Gen3: 2

Gen4: 2

Indicates the RX CTLE pole value.
RX VGA1 (For P-Tile debug toolkit only)

Gen3: 5

Gen4: 5

Indicates the RX AFE first stage VGA gain value.
RX VGA2 (For P-Tile debug toolkit only)

Gen3: 5

Gen4: 5

Indicates the RX AFE second stage VGA gain value.
DFE Enable (For P-Tile debug toolkit only) Enable, Disable

Indicates DFE adaptation is enabled for taps 1 - 5.

Enable: DFE adaptation is enabled for taps 1 - 5.

Disable: DFE adaptation is disabled for taps 1 - 5.
DFE Tap1 adapted value (For P-Tile debug toolkit only) <-128 to 127>

Indicates the adapted value of DFE tap 1. This is a signed input (two's complement encoded).
DFE Tap2 adapted value (For P-Tile debug toolkit only) <-32 to 31>

Indicates the adapted value of DFE tap 2. This is a signed input (two's complement encoded).
DFE Tap3 adapted value (For P-Tile debug toolkit only) <-32 to 31>

Indicates the adapted value of DFE tap 3. This is a signed input (two's complement encoded).
DFE Tap4 adapted value (For P-Tile debug toolkit only) <-32 to 31>

Indicates the adapted value of DFE tap 4. This is a signed input (two's complement encoded).
DFE Tap5 adapted value (For P-Tile debug toolkit only) <-32 to 31>

Indicates the adapted value of DFE tap 5. This is a signed input (two's complement encoded).

RX VGA Gain (For F-Tile debug toolkit only)

 <0 to 127>

Indicates the RX AFE VGA gain value.
RX High Freq Boost (For F-Tile debug toolkit only) <0 to 63>

Indicates the RX AFE high frequency boost value.
DFE Data Tap1 (For F-Tile debug toolkit only) <0 to 63>

Indicates the adapted value of DFE tap 1.
Figure 69. Example of Receiver Settings
2 Use Clear event counter
Level Two Title