Multi Channel DMA Intel® FPGA IP for PCI Express User Guide

Download PDF
ID 683821
Date 10/29/2021
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Terms and Acronyms 2. Introduction 3. Functional Description 4. Interface Overview 5. Parameters (H-Tile) 6. Parameters (P-Tile and F-Tile) 7. Designing with the IP Core 8. Software Programming Model 9. Registers 10. Troubleshooting/Debugging 11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives 12. Revision History for Multi Channel DMA Intel FPGA IP for PCI Express User Guide
2. Introduction x
2.1. Multi Channel DMA IP for PCI Express Features 2.2. Device Family Support 2.3. Recommended Speed Grades 2.4. Resource Utilization 2.5. Release Information
2.1. Multi Channel DMA IP for PCI Express Features x
2.1.1. Endpoint Mode 2.1.2. Root Port Mode
3. Functional Description x
3.1. Multi Channel DMA 3.2. Bursting Avalon-MM Master (BAM) 3.3. Bursting Avalon-MM Slave (BAS) 3.4. Config Slave 3.5. Hard IP Reconfiguration Interface 3.6. Config TL Interface 3.7. Configuration Intercept Interface (EP Only)
3.1. Multi Channel DMA x
3.1.1. H2D Data Mover 3.1.2. D2H Data Mover 3.1.3. Descriptors 3.1.4. Avalon-MM PIO Master 3.1.5. Avalon-MM Write (H2D) and Read (D2H) Master 3.1.6. Avalon-ST Source (H2D) and Sink (D2H) 3.1.7. User MSI-X 3.1.8. User Functional Level Reset (FLR) 3.1.9. Control Registers
3.1.2. D2H Data Mover x
3.1.2.1. D2H Descriptor Fetch
3.1.3. Descriptors x
3.1.3.1. Metadata Support 3.1.3.2. MSI-X/Writeback
3.1.6. Avalon-ST Source (H2D) and Sink (D2H) x
3.1.6.1. Avalon-ST 4-Port mode 3.1.6.2. Avalon-ST 1-Port Mode 3.1.6.3. Packet (File) Boundary 3.1.6.4. Metadata
4. Interface Overview x
4.1. Port List 4.2. Clocks 4.3. Resets 4.4. Multi Channel DMA 4.5. Bursting Avalon-MM Master (BAM) Interface 4.6. Bursting Avalon-MM Slave (BAS) Interface 4.7. Config Slave Interface (RP only) 4.8. Hard IP Reconfiguration Interface 4.9. Config TL Interface 4.10. Configuration Intercept Interface (EP Only) 4.11. User Functional Level Reset (FLR) 4.12. User Event MSI-X Request Interface
4.1. Port List x
4.1.1. Port List (H-Tile) 4.1.2. Port List (P-Tile and F-Tile)
4.4. Multi Channel DMA x
4.4.1. Avalon-MM PIO Master 4.4.2. Avalon-MM Write Master (H2D) 4.4.3. Avalon-MM Read Master (D2H) 4.4.4. Avalon-ST Source (H2D) 4.4.5. Avalon-ST Sink (D2H) 4.4.6. User MSI-X Interface 4.4.7. User FLR Interface
5. Parameters (H-Tile) x
5.1. IP Settings 5.2. Example Designs
5.1. IP Settings x
5.1.1. System Settings 5.1.2. MCDMA Settings 5.1.3. Device Identification Registers 5.1.4. Multifunction and SR-IOV System Settings Parameters [Endpoint Mode] 5.1.5. Configuration, Debug and Extension Options 5.1.6. PHY Characteristics 5.1.7. PCI Express / PCI Capabilities Parameters
5.1.7. PCI Express / PCI Capabilities Parameters x
5.1.7.1. Device 5.1.7.2. Link 5.1.7.3. MSI-X 5.1.7.4. Power Management 5.1.7.5. Vendor Specific Extended Capability (VSEC)
6. Parameters (P-Tile and F-Tile) x
6.1. IP Settings 6.2. PCIe0 Settings 6.3. Example Designs
6.1. IP Settings x
6.1.1. Top-Level Settings
6.2. PCIe0 Settings x
6.2.1. Base Address Register 6.2.2. PCIe0 Configuration, Debug and Extension Options 6.2.3. PCIe0 Device Identification Registers 6.2.4. PCIe0 PCI Express / PCI Capabilities Parameters 6.2.5. MCDMA Settings
6.2.3. PCIe0 Device Identification Registers x
6.2.3.1. PCIe0 PF0 IDs 6.2.3.2. PCIe0 PF0 VF IDs
6.2.4. PCIe0 PCI Express / PCI Capabilities Parameters x
6.2.4.1. PCIe0 Device 6.2.4.2. PCIe0 Link 6.2.4.3. PCIe0 MSI-X 6.2.4.4. PCIe0 DEV SER 6.2.4.5. PCIe0 PRS 6.2.4.6. PCIe0 ACS Capabilities 6.2.4.7. PCIe0 ATS 6.2.4.8. PCIe0 TPH 6.2.4.9. PCIe0 VIRTIO
6.2.4.6. PCIe0 ACS Capabilities x
6.2.4.6.1. PCIe0 ACS for Physical Functions 6.2.4.6.2. PCIe0 ACS for Virtual Functions
6.2.4.7. PCIe0 ATS x
6.2.4.7.1. PCIe0 ATS for Physical Functions 6.2.4.7.2. PCIe0 ATS for Virtual Functions
6.2.4.8. PCIe0 TPH x
6.2.4.8.1. PCIe0 TPH for Physical Functions 6.2.4.8.2. PCIe0 TPH for Virtual Functions
7. Designing with the IP Core x
7.1. Generating the IP Core 7.2. Simulating the IP Core 7.3. IP Core Generation Output - Intel® Quartus® Prime Pro Edition 7.4. Systems Integration and Implementation
7.4. Systems Integration and Implementation x
7.4.1. Required Supporting IP
8. Software Programming Model x
8.1. Multi Channel DMA IP Custom Driver 8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver 8.3. Multi Channel DMA IP Kernel Mode Character Device Driver 8.4. Multi Channel DMA IP Kernel Mode Network Device Driver
8.1. Multi Channel DMA IP Custom Driver x
8.1.1. Architecture 8.1.2. libmqdma library details 8.1.3. Application 8.1.4. Software Flow 8.1.5. API Flow 8.1.6. libmqdma Library API List 8.1.7. Request Structures
8.1.2. libmqdma library details x
8.1.2.1. Channel Initialization 8.1.2.2. Descriptor Memory Management 8.1.2.3. Descriptor Completion Status Update
8.1.5. API Flow x
8.1.5.1. Single Descriptor Load and Submit 8.1.5.2. Multiple Descriptor Load and Submit
8.1.6. libmqdma Library API List x
8.1.6.1. ifc_api_start 8.1.6.2. ifc_mcdma_port_by_name 8.1.6.3. ifc_qdma_device_get 8.1.6.4. ifc_num_channels_get 8.1.6.5. ifc_qdma_channel_get 8.1.6.6. ifc_qdma_acquire_channels 8.1.6.7. ifc_qdma_release_all_channels 8.1.6.8. ifc_qdma_device_put 8.1.6.9. ifc_qdma_channel_put 8.1.6.10. ifc_qdma_completion_poll 8.1.6.11. ifc_qdma_request_start 8.1.6.12. ifc_qdma_request_prepare 8.1.6.13. ifc_qdma_descq_queue_batch_load 8.1.6.14. ifc_qdma_request_submit 8.1.6.15. ifc_qdma_pio_read32 8.1.6.16. ifc_qdma_pio_write32 8.1.6.17. ifc_qdma_pio_read64 8.1.6.18. ifc_qdma_pio_write64 8.1.6.19. ifc_qdma_pio_read128 8.1.6.20. ifc_qdma_pio_write128 8.1.6.21. ifc_qdma_pio_read256 8.1.6.22. ifc_qdma_pio_write256 8.1.6.23. ifc_request_malloc 8.1.6.24. ifc_request_free 8.1.6.25. ifc_app_stop 8.1.6.26. ifc_qdma_poll_init 8.1.6.27. ifc_qdma_poll_add 8.1.6.28. ifc_qdma_poll_wait 8.1.6.29. ifc_mcdma_port_by_name
8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver x
8.2.1. Architecture 8.2.2. MCDMA Poll Mode Driver 8.2.3. DPDK based application 8.2.4. Request Structures 8.2.5. Software Flow 8.2.6. API Flow 8.2.7. API List
8.2.2. MCDMA Poll Mode Driver x
8.2.2.1. Completion Status Update 8.2.2.2. Metadata Support 8.2.2.3. User MSI-X
8.3. Multi Channel DMA IP Kernel Mode Character Device Driver x
8.3.1. Architecture 8.3.2. libmcmem Library Information 8.3.3. Kernel Driver Information 8.3.4. Control Message Structure 8.3.5. Software Flow 8.3.6. API Flow 8.3.7. API List
8.3.3. Kernel Driver Information x
8.3.3.1. Device Management IP Reset Open Close Read & Write Readv & Writev 8.3.3.2. Channel Management 8.3.3.3. Completions Management
8.3.3.3. Completions Management x
8.3.3.3.1. Interrupt Mode 8.3.3.3.2. Poll Mode
8.4. Multi Channel DMA IP Kernel Mode Network Device Driver x
8.4.1. Architecture 8.4.2. Driver Information 8.4.3. Software Flow
8.4.2. Driver Information x
8.4.2.1. Device Management 8.4.2.2. Channel Management 8.4.2.3. Descriptor Memory Management 8.4.2.4. Completions Management
8.4.3. Software Flow x
8.4.3.1. Host to Device Flow 8.4.3.2. Device to Host Flow
9. Registers x
9.1. Queue Control (QCSR) 9.2. MSI-X Memory Space 9.3. Control Register (GCSR)
10. Troubleshooting/Debugging x
10.1. Debug Toolkit
10.1. Debug Toolkit x
10.1.1. Overview 10.1.2. Enabling the P-Tile Debug Toolkit 10.1.3. Launching the P-Tile Debug Toolkit 10.1.4. Using the P-Tile Debug Toolkit
10.1.4. Using the P-Tile Debug Toolkit x
10.1.4.1. Main View 10.1.4.2. Toolkit Parameters 10.1.4.3. Channel Parameters 10.1.4.4. Eye Viewer
1. Terms and Acronyms
2. Introduction
3. Functional Description
4. Interface Overview
5. Parameters (H-Tile)
6. Parameters (P-Tile and F-Tile)
7. Designing with the IP Core
8. Software Programming Model
9. Registers
10. Troubleshooting/Debugging
11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives
12. Revision History for Multi Channel DMA Intel FPGA IP for PCI Express User Guide

8.3.3.1. Device Management

IP Reset

IP reset is performed through the sysfs attribute file mcdma_ipreset. Following command performs the IP reset: 
echo 1 > /sys/bus/pci/devices/<bdf>/mcdma_ipreset

Open

When the userspace application calls the open system call on the char dev node, kernel module checks and allocates an available channel to that application and stores the channel context in a private area of the file descriptor context, and the file descriptor is returned. The same file descriptor is used for further channel-specific operations.

Close

Once all the DMA transactions are completed, you must close the file allocated during opening of the device. As part of this, kernel retrieves the channel context associated to that device and frees the same to device.

Figure 41. Open/Close

Read & Write

When the userspace application wants to perform DMA, it is needed to call standard read/write system calls by providing the pre-allocated DMA memory acquired from the libmcmem library. The application must provide the physical starting address of the buffer. The kernel module converts the DMA buffer into a Hardware specified descriptor and submits it.

Figure 42. Read/Write

Readv & Writev

Readv & writev system calls are used to submit DMA requests in batches.

Level Two Title