Multi Channel DMA Intel® FPGA IP for PCI Express User Guide

Download PDF
ID 683821
Date 4/11/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Before You Begin 2. Introduction 3. Functional Description 4. Interface Overview 5. Parameters (H-Tile) 6. Parameters (P-Tile) (F-Tile) (R-Tile) 7. Designing with the IP Core 8. Software Programming Model 9. Registers 10. Troubleshooting/Debugging 11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives 12. Revision History for Multi Channel DMA Intel FPGA IP for PCI Express User Guide
1. Before You Begin x
1.1. Terms and Acronyms 1.2. Known Issues
2. Introduction x
2.1. Multi Channel DMA IP for PCI Express Features 2.2. Device Family Support 2.3. Recommended Speed Grades 2.4. Resource Utilization 2.5. Release Information
2.1. Multi Channel DMA IP for PCI Express Features x
2.1.1. Endpoint Mode 2.1.2. Root Port Mode
3. Functional Description x
3.1. Multi Channel DMA 3.2. Bursting Avalon-MM Master (BAM) 3.3. Bursting Avalon-MM Slave (BAS) 3.4. MSI Interrupt 3.5. Config Slave (CS) 3.6. Root Port Address Translation Table Enablement 3.7. Hard IP Reconfiguration Interface 3.8. Config TL Interface 3.9. Configuration Intercept Interface (EP Only) 3.10. Data Mover Only
3.1. Multi Channel DMA x
3.1.1. H2D Data Mover 3.1.2. D2H Data Mover 3.1.3. Descriptors 3.1.4. Avalon-MM PIO Master 3.1.5. Avalon-MM Write (H2D) and Read (D2H) Master 3.1.6. Avalon-ST Source (H2D) and Sink (D2H) 3.1.7. User MSI-X 3.1.8. User Functional Level Reset (FLR) 3.1.9. Control Registers
3.1.2. D2H Data Mover x
3.1.2.1. D2H Descriptor Fetch
3.1.3. Descriptors x
3.1.3.1. Support for Unaligned (or byte-aligned) Data Transfer 3.1.3.2. Metadata Support 3.1.3.3. MSI-X/Writeback
3.1.6. Avalon-ST Source (H2D) and Sink (D2H) x
3.1.6.1. Avalon-ST 1-Port Mode 3.1.6.2. Packet (File) Boundary 3.1.6.3. Metadata
3.4. MSI Interrupt x
3.4.1. Endpoint MSI Support through BAS 3.4.2. MSI Interrupt Controller
3.5. Config Slave (CS) x
3.5.1. 29 bit AVMM address format 3.5.2. 14 bit AVMM address format 3.5.3. Configuration Access Mechanism
3.10. Data Mover Only x
3.10.1. H2D Data Mover 3.10.2. D2H Data Mover 3.10.3. Application Specific Bits
3.10.1. H2D Data Mover x
3.10.1.1. H2D Descriptor Format (h2ddm_desc) 3.10.1.2. H2D Descriptor Completion Packet Format (h2ddm_desc_cmpl) 3.10.1.3. H2D Descriptor Status (h2ddm_desc_status)
3.10.2. D2H Data Mover x
3.10.2.1. D2H Descriptor Format (d2hdm_desc) 3.10.2.2. D2H Descriptor Status (d2hdm_desc_status)
4. Interface Overview x
4.1. Port List 4.2. Clocks 4.3. Resets 4.4. Multi Channel DMA 4.5. Bursting Avalon-MM Master (BAM) Interface 4.6. Bursting Avalon-MM Slave (BAS) Interface 4.7. MSI Interface 4.8. Config Slave Interface (RP only) 4.9. Hard IP Reconfiguration Interface 4.10. Config TL Interface 4.11. Configuration Intercept Interface (EP Only) 4.12. Data Mover Interface 4.13. Hard IP Status Interface
4.1. Port List x
4.1.1. Port List (H-Tile) 4.1.2. Port List (P-Tile) (F-Tile) (R-Tile)
4.2. Clocks x
4.2.1. F-Tile System PLL Reference Clock Requirements
4.4. Multi Channel DMA x
4.4.1. Avalon-MM PIO Master 4.4.2. Avalon-MM Write Master (H2D) 4.4.3. Avalon-MM Read Master (D2H) 4.4.4. Avalon-ST Source (H2D) 4.4.5. Avalon-ST Sink (D2H) 4.4.6. User Event MSI-X Interface 4.4.7. User Functional Level Reset (FLR) Interface
4.12. Data Mover Interface x
4.12.1. H2D Data Mover Interface 4.12.2. D2H Data Mover Interface
5. Parameters (H-Tile) x
5.1. IP Settings 5.2. Example Designs
5.1. IP Settings x
5.1.1. System Settings 5.1.2. MCDMA Settings 5.1.3. Device Identification Registers 5.1.4. Multifunction and SR-IOV System Settings Parameters [Endpoint Mode] 5.1.5. Configuration, Debug and Extension Options 5.1.6. PHY Characteristics 5.1.7. PCI Express / PCI Capabilities Parameters
5.1.7. PCI Express / PCI Capabilities Parameters x
5.1.7.1. Device 5.1.7.2. Link 5.1.7.3. MSI-X 5.1.7.4. Power Management 5.1.7.5. Vendor Specific Extended Capability (VSEC)
6. Parameters (P-Tile) (F-Tile) (R-Tile) x
6.1. Top-Level Settings 6.2. PCIe0 Settings 6.3. PCIe1 Settings 6.4. Example Designs
6.2. PCIe0 Settings x
6.2.1. Base Address Register 6.2.2. PCIe0 Configuration, Debug and Extension Options 6.2.3. PCIe0 Device Identification Registers 6.2.4. PCIe0 PCI Express / PCI Capabilities Parameters 6.2.5. MCDMA Settings
6.2.3. PCIe0 Device Identification Registers x
6.2.3.1. PCIe0 PF0 IDs 6.2.3.2. PCIe0 PF0 VF IDs
6.2.4. PCIe0 PCI Express / PCI Capabilities Parameters x
6.2.4.1. PCIe0 Device 6.2.4.2. PCIe0 Link 6.2.4.3. PCIe0 MSI-X 6.2.4.4. PCIe0 DEV SER 6.2.4.5. PCIe0 PRS 6.2.4.6. PCIe0 ACS Capabilities 6.2.4.7. PCIe0 ATS 6.2.4.8. PCIe0 TPH
6.2.4.6. PCIe0 ACS Capabilities x
6.2.4.6.1. PCIe0 ACS for Physical Functions 6.2.4.6.2. PCIe0 ACS for Virtual Functions
6.2.4.7. PCIe0 ATS x
6.2.4.7.1. PCIe0 ATS for Physical Functions 6.2.4.7.2. PCIe0 ATS for Virtual Functions
6.2.4.8. PCIe0 TPH x
6.2.4.8.1. PCIe0 TPH for Physical Functions 6.2.4.8.2. PCIe0 TPH for Virtual Functions
6.3. PCIe1 Settings x
6.3.1. PCIe1 Configuration, Debug and Extension Options
7. Designing with the IP Core x
7.1. Generating the IP Core 7.2. Simulating the IP Core 7.3. IP Core Generation Output - Intel® Quartus® Prime Pro Edition 7.4. Systems Integration and Implementation
7.4. Systems Integration and Implementation x
7.4.1. Required Supporting IP
8. Software Programming Model x
8.1. Multi Channel DMA IP Custom Driver 8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver 8.3. Multi Channel DMA IP Kernel Mode Network Device Driver
8.1. Multi Channel DMA IP Custom Driver x
8.1.1. Architecture 8.1.2. libmqdma library details 8.1.3. Application 8.1.4. Software Flow 8.1.5. API Flow 8.1.6. libmqdma Library API List 8.1.7. Request Structures
8.1.2. libmqdma library details x
8.1.2.1. Channel Initialization 8.1.2.2. Descriptor Memory Management 8.1.2.3. Descriptor Completion Status Update
8.1.5. API Flow x
8.1.5.1. Single Descriptor Load and Submit 8.1.5.2. Multiple Descriptor Load and Submit
8.1.6. libmqdma Library API List x
8.1.6.1. ifc_api_start 8.1.6.2. ifc_mcdma_port_by_name 8.1.6.3. ifc_qdma_device_get 8.1.6.4. ifc_num_channels_get 8.1.6.5. ifc_qdma_channel_get 8.1.6.6. ifc_qdma_acquire_channels 8.1.6.7. ifc_qdma_release_all_channels 8.1.6.8. ifc_qdma_device_put 8.1.6.9. ifc_qdma_channel_put 8.1.6.10. ifc_qdma_completion_poll 8.1.6.11. ifc_qdma_request_start 8.1.6.12. ifc_qdma_request_prepare 8.1.6.13. ifc_qdma_descq_queue_batch_load 8.1.6.14. ifc_qdma_request_submit 8.1.6.15. ifc_qdma_pio_read32 8.1.6.16. ifc_qdma_pio_write32 8.1.6.17. ifc_qdma_pio_read64 8.1.6.18. ifc_qdma_pio_write64 8.1.6.19. ifc_qdma_pio_read128 8.1.6.20. ifc_qdma_pio_write128 8.1.6.21. ifc_qdma_pio_read256 8.1.6.22. ifc_qdma_pio_write256 8.1.6.23. ifc_request_malloc 8.1.6.24. ifc_request_free 8.1.6.25. ifc_app_stop 8.1.6.26. ifc_qdma_poll_init 8.1.6.27. ifc_qdma_poll_add 8.1.6.28. ifc_qdma_poll_wait 8.1.6.29. ifc_mcdma_port_by_name
8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver x
8.2.1. Architecture 8.2.2. MCDMA Poll Mode Driver 8.2.3. DPDK based application 8.2.4. Request Structures 8.2.5. Software Flow 8.2.6. API Flow 8.2.7. API List
8.2.2. MCDMA Poll Mode Driver x
8.2.2.1. Completion Status Update 8.2.2.2. Metadata Support 8.2.2.3. User MSI-X
8.3. Multi Channel DMA IP Kernel Mode Network Device Driver x
8.3.1. Architecture 8.3.2. Driver Information 8.3.3. Software Flow
8.3.2. Driver Information x
8.3.2.1. Device Management 8.3.2.2. Channel Management 8.3.2.3. Descriptor Memory Management 8.3.2.4. Completions Management 8.3.2.5. ethtool support 8.3.2.6. debugfs support 8.3.2.7. SRIOV Support
8.3.3. Software Flow x
8.3.3.1. Host to Device Flow 8.3.3.2. Device to Host Flow
9. Registers x
9.1. Queue Control (QCSR) 9.2. MSI-X Memory Space 9.3. Control Register (GCSR)
10. Troubleshooting/Debugging x
10.1. Debug Toolkit
10.1. Debug Toolkit x
10.1.1. Overview 10.1.2. Enabling the Debug Toolkit 10.1.3. Launching the Debug Toolkit 10.1.4. Using the Debug Toolkit
10.1.4. Using the Debug Toolkit x
10.1.4.1. Main View 10.1.4.2. Toolkit Parameters 10.1.4.3. Channel Parameters 10.1.4.4. Eye Viewer 10.1.4.5. Link Inspector
10.1.4.4. Eye Viewer x
10.1.4.4.1. Running Eye Viewer in the P-Tile Debug Toolkit 10.1.4.4.2. Running Eye Viewer in the F-Tile Debug Toolkit
1. Before You Begin
2. Introduction
3. Functional Description
4. Interface Overview
5. Parameters (H-Tile)
6. Parameters (P-Tile) (F-Tile) (R-Tile)
7. Designing with the IP Core
8. Software Programming Model
9. Registers
10. Troubleshooting/Debugging
11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives
12. Revision History for Multi Channel DMA Intel FPGA IP for PCI Express User Guide

8.2.7. API List

Table 122.  rte_eth_dev_configure
API API Description Input Parameters Return Values
int rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, const struct rte_eth_conf *dev_conf)

This API configures an Ethernet device. This function must be invoked first before any other function in the Ethernet API.

port ID: device ID

nb_tx_queues : Number of TX Queues

num_rx_queues: Number of Rx Queues

eth_config : input configuration

0 Success, device configured

<0 Error code returned by the driver configuration function

Table 123.  rte_eth_tx_queue_setup
API API Description Input Parameters Return Values

Int rte_eth_tx_queue_setup(uint16_t port_id, uint16_t tx_queue_id, uint16_t nb_tx_desc, unsigned int socket_id, const struct rte_eth_rxconf *tx_conf)

This API allocates and set up a receive queue for DMA device.

port ID: Port ID of device

tx_queue_id: Queue ID

nb_tx_desc: Number of Tx descriptors to allocate for the transmit ring

socket_id: socket identifier

tx_conf: TX configuration context

0 on success

negative otherwise

Table 124.  rte_eth_rx_queue_setup
API API Description Input Parameters Return Values

int rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id, uint16_t nb_rx_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp)

This API allocates and set up a receive queue for DMA device.

port ID: Port ID of device

rx_queue_id: Queue ID

nb_rx_desc: Number of Rx descriptors to allocate for the receive ring

socket_id: socket identifier

rx_conf: RX configuration context

mp: Pointer to the memory pool from which is used to allocate memory buffers for each descriptor of the receive ring

0 on success

negative otherwise
Table 125.  rte_eth_dev_set_mtu
API API Description Input Parameters Return Values

int rte_eth_dev_set_mtu(uint16_t port_id, uint16_t mtu)

This API sets the payload value for the processing.

port ID: Port ID of device

mtu: MTU to be applied

0 on success

negative otherwise

Table 126.  rte_eth_dev_start
API API Description Input Parameters Return Values

int rte_eth_dev_start(uint16_t port_id)

This API starts the Ethernet device by initializing descriptors, QCSR Rx and Tx context.

port ID: Port ID of device

0 on success

negative otherwise
Table 127.  rte_eth_dev_stop
API API Description Input Parameters Return Values

void rte_eth_dev_stop(uint16_t port_id)

This API stops the Ethernet device.

port ID: Port ID of device

void
Table 128.  rte_eth_dev_close
API API Description Input Parameters Return Values

Void rte_eth_dev_close(uint16_t port_id)

This API closes the Ethernet device.

port ID: Port ID of device

void
Table 129.  rte_eth_tx_burst
API API Description Input Parameters Return Values

static inline uint16_t rte_eth_tx_burst (uint16_t port_id, uint16_t queue_id, struct rte_mbuf **tx_pkts, const uint16_t nb_pkts)

This API is used to transmit burst of packets.

port ID: Port ID of device queue_id:

Queue ID tx_pkts: Array of pointers to *rte_mbuf* structures

nb_pkts: Maximum number of packets to retrieve

The number of output packets actually stored in transmit descriptors.

Table 130.  rte_eth_rx_burst
API API Description Input Parameters Return Values

static inline uint16_t rte_eth_rx_burst (uint16_t port_id, uint16_t queue_id, struct rte_mbuf **rx_pkts, const uint16_t nb_pkts)

This API is used to receives burst of packets.

port ID: Port ID of device

queue_id: Queue ID rx_pkts: Array of pointers to *rte_mbuf* structures

nb_pkts: Maximum number of packets to retrieve

The number of packets actually retrieved.
Table 131.   ifc_mcdma_pio_read64
API API Description Input Parameters Return Values

Uint64_t ifc_mcdma_pio_read64(struct ifc_mcdma_device *qdev, uint64_t addr); - New

Read 64b value from BAR2 address This API is used for PIO testing, dumping statistics, pattern generation etc.

qdev: MCDMA device

addr: address to read

0 on success

negative otherwise
Table 132.   ifc_mcdma_pio_write64
API API Description Input Parameters Return Values

void ifc_mcdma_pio_write64(struct ifc_mcdma_device *qdev, uint64_t addr, uint64_t val) - New

Writes 64 bit value to BAR2 address

qdev: MCDMA device addr: address to write

val: value to write

0 on success and populates channel context

negative otherwise
Table 133.   ifc_mcdma_pio_read128
API API Description Input Parameters Return Values

int ifc_mcdma_pio_read128(uint16_t portid, uint64_t offset, uint64_t *buf, int bar_num) - New

Read 128 bit value from specified address of specified BAR

This API is used for PIO testing etc.

qdev: MCDMA device

addr: address to read

0 on success

negative otherwise
Table 134.   ifc_mcdma_pio_write128
API API Description Input Parameters Return Values

int ifc_mcdma_pio_write128(uint16_t portid, uint64_t offset, uint64_t *val, int bar_num) – New

Writes 128 bit value to specified address of specified BAR.

This API is used for PIO testing etc.

qdev: MCDMA device

addr: address to write

val: value to write

0 on success and populates channel context

negative otherwise
Table 135.   ifc_mcdma_pio_read256
API API Description Input Parameters Return Values

int ifc_mcdma_pio_read256(uint16_t portid, uint64_t offset, uint64_t *buf, int bar_num) – New

Read 256 bit value from specified address of specified BAR

This API is used for PIO testing etc.

qdev: MCDMA device

addr: address to read

0 on success

negative otherwise
Table 136.   ifc_mcdma_pio_write256
API API Description Input Parameters Return Values

int ifc_mcdma_pio_write512(uint16_t portid, uint64_t offset, uint64_t *val, int bar_num) – New

Writes 128 bit value to specified address of specified BAR

This API is used for PIO testing etc.

qdev: MCDMA device

addr: address to write

val: value to write

0 on success and populates channel context

negative otherwise
Level Two Title