MACsec Intel® FPGA System Design User Guide

Download PDF
ID 767516
Date 10/02/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Introduction 2. Architecture 3. Interface Overview 4. Parameters 5. Configuration Registers 6. Software Architecture 7. Generating the System Design 8. Document Revision History for MACsec System Design User Guide A. Release Notes
1. Introduction x
1.1. Terminology 1.2. Release Information 1.3. System Requirements
2. Architecture x
2.1. System Architecture 2.2. Data Path Between Ethernet MAC and MACsec 2.3. Data Path Between MACsec and MCDMA 2.4. Data Path Between MACsec and Packet Generator/Checker (Packet Client) 2.5. Data Path Illustrations 2.6. Interrupts 2.7. Packet FIFO 2.8. AXI-ST Rate Controller 2.9. Error Handling 2.10. Top Level Signals
2.2. Data Path Between Ethernet MAC and MACsec x
2.2.1. Ethernet Subsystem
2.2.1. Ethernet Subsystem x
2.2.1.1. Data Receiving from MACsec to E/F-Tile Hard IP 2.2.1.2. Data Transmitting from E/F-tile Hard IP to MACsec 2.2.1.3. Order of Ethernet Transmission 2.2.1.4. E/F-Tile Hard IP Reset Sequence
2.3. Data Path Between MACsec and MCDMA x
2.3.1. AXI-ST Multi-Segment to Single-Segment Conversion 2.3.2. MCDMA
2.4. Data Path Between MACsec and Packet Generator/Checker (Packet Client) x
2.4.1. Packet Client 2.4.2. Packet Generation and Check
2.5. Data Path Illustrations x
2.5.1. MACsec Interface Signal Names
2.6. Interrupts x
2.6.1. MCDMA MSI-X Table Configuration
3. Interface Overview x
3.1. Clocking 3.2. Resets
5. Configuration Registers x
5.1. System Register Map
5.1. System Register Map x
5.1.1. Packet Client Register Map 5.1.2. Interrupt Controller Register Map
6. Software Architecture x
6.1. MACsec Key Agreement Protocol 6.2. Driver Functional Requirements 6.3. Software Requirements 6.4. Software Overview 6.5. MACsec IP APIs Sequence 6.6. Functions 6.7. Software Tools
6.4. Software Overview x
6.4.1. McDMA Driver Kernel Module 6.4.2. MACsec IP APIs 6.4.3. Linux MACsec Driver Kernel Module 6.4.4. IP Tool 6.4.5. WPA Supplicant
6.4.2. MACsec IP APIs x
6.4.2.1. SDK
6.5. MACsec IP APIs Sequence x
6.5.1. MACsec Initialization Sequence
6.5.1. MACsec Initialization Sequence x
6.5.1.1. MACsec Reset Sequence 6.5.1.2. TX Configuration Sequence 6.5.1.3. RX Configuration Sequence 6.5.1.4. TX Rekeying Sequence 6.5.1.5. RX Rekeying Sequence 6.5.1.6. Cut Through/Store Forward Mode 6.5.1.7. User Single/Multi Port Settings 6.5.1.8. Encrypt/Decrypt Port 6.5.1.9. Port Priority 6.5.1.10. Interrupt Generation and Register
6.5.1.1. MACsec Reset Sequence x
6.5.1.1.1. GLOBAL Reset 6.5.1.1.2. PORT Reset 6.5.1.1.3. Secure Association (SA) Reset
6.6. Functions x
6.6.1. macsec_initilize 6.6.2. macsec_get_attributes 6.6.3. macsec_get_sa_attributes 6.6.4. macsec_set_attributes 6.6.5. macsec_set_sa_attributes 6.6.6. macsec_read_register 6.6.7. macsec_write_register 6.6.8. macsec_set_port_configuration 6.6.9. macsec_rate_configuration 6.6.10. macsec_single_or_multi_port 6.6.11. macsec_crypto_mode 6.6.12. macsec_port_priority 6.6.13. macsec_register_isr
6.7. Software Tools x
6.7.1. IP Tool 6.7.2. WPA_Supplicant 6.7.3. CLI Debug Tool
6.7.3. CLI Debug Tool x
6.7.3.1. Functional Requirements 6.7.3.2. Features Overview 6.7.3.3. Design with HOST 6.7.3.4. Netlink Interface 6.7.3.5. SDK 6.7.3.6. Design Overview
7. Generating the System Design x
7.1. Software Requirements 7.2. Obtaining the Reference Design 7.3. Reference Design Directory Structure 7.4. Simulation Command Arguments 7.5. Simulation Test Cases 7.6. Complete Simulation Command 7.7. Simulation Requirements 7.8. Running Non-UVM Simulation 7.9. Running UVM Simulation 7.10. Building, Installing, and Running the Software 7.11. Building the Hardware Design
7.4. Simulation Command Arguments x
7.4.1. Simulation -d Options
1. Introduction
2. Architecture
3. Interface Overview
4. Parameters
5. Configuration Registers
6. Software Architecture
7. Generating the System Design
8. Document Revision History for MACsec System Design User Guide
A. Release Notes

2.3.2. MCDMA

The Multi Channel DMA for PCI Express enables you to efficiently transfer data between the host and device. It supports multiple DMA channels between the host and device over the underlying PCIe link. A DMA channel consists of an H2D (host to device) and D2H (device to host) queue pair.

As shown in the figure below, the Multi Channel DMA can be used in a server’s hardware infrastructure to allow communication between various VM clients and their FPGA-device based counterparts. The Multi Channel DMA operates on descriptor-based queues set up by driver software to transfer data between local FPGA and the host. The Multi Channel DMA ’s control logic reads the queue descriptors and executes them.

The Multi Channel DMA IP integrates the Intel® PCIe Hard IP and interfaces with the host Root Complex via the PCIe serial lanes. On the user logic interface, the Avalon-MM/Avalon-ST interfaces allow the designer easy integration of the multi Channel DMA IP for PCI Express with other Platform Designer components.
Figure 17. MCDMA
The MCDMA engine operates on a software DMA queue to transfer data between the local FPGA and the host. The elements of each queue are software descriptors that are written by driver/software. Hardware reads the queue descriptors and executes them. Data Mover Blocks for both the directions (D2H, H2D) fetch the descriptors and are responsible for data transfers to or from the system memory locations specified in the descriptors or from/to the user application in the hardware.
Figure 18. Descriptor Linked-List

A DMA channel to support Multi Channel DMA data movement consists of a pair of the descriptor queues: one H2D descriptor queue and one D2H descriptor queue. As shown in the figure above, the descriptors are arranged contiguously within a 4 KB page. Each descriptor is 32 bytes in size. The descriptors are kept in host memory in a linked-list of 4 KB pages. For a 32 byte descriptor and a 4 KB page, each page contains up to 128 descriptors. The last descriptor in a 4 KB page must be a “link descriptor” – a descriptor containing a link to the next 4 KB page with the link bit set to 1. The last entry in the linked list must be a link pointing to the base address programmed in the QCSR, in order to achieve a circular buffer containing a linked-list of 4 KB pages.

Software and hardware communicate and manage the descriptors using tail index pointer (Q_TAIL_POINTER) and head index pointer (Q_HEAD_POINTER) QCSR registers as shown in the figure below. The DMA starts when software writes the last valid descriptor index to the Q_TAIL_POINTER register.
Figure 19. Buffer Descriptor (BD) Ring

Descriptors between the Q_HEAD_POINTER and Q_TAIL_POINTER are enabled for HW usage and descriptors between the Q_TAIL_POINTER and Q_HEAD_POINTER are in SW control. DMA operations for a particular queue pause when both the pointers are same.

It also offers a DMA-bypass capability to the host for doing PIO Read/Writes to device memory. This interface is used for downstream logic CSR access. PCIe BAR2 is mapped to the Avalon-MM PIO Master. Any TLP targeting BAR2 is forwarded to the user logic. TLP addresses targeting the PIO interface should be 8 bytes aligned. The PIO interface supports non-bursting 64-bit write and read transfers.

The PIO interface address mapping is as follows: PIO address = {vf_active, pf, vf, csr_addr}
  1. vf_active: This indicates that SRIOV is enabled.
  2. pf [PF_NUM-1:0]: Physical function number decoded from the PCIe header received from the HIP; PF_NUM which is ($clog2(pf_num_tcl)) is the RTL design parameter selected by you such that Multi Channel DMA IP only allocates required number of the bits on Avalon-MM side to limit the number of the wires on the user interface.
  3. vf [VF_NUM-1:0]: Virtual function number decoded from the PCIe header received from the HIP; VF_NUM which is ($clog2(vf_num_tcl)) is the RTL design parameter selected by you such that Multi Channel DMA IP only allocates required number of the bits on Avalon-MM side to limit the number of the wires on the user interface.
  4. csr_addr [ADDR_SIZE-1:0]: Number of bits required for BAR2 size requested across all Functions (PFs and VFs) Example: If BAR2 is selected as 4 MB, the ADDR_SIZE = 22.
The table below shows the DMA channel mapping w.r.t MCDMA IP parameter setting.
  • Number of PFs – 2
  • Number of VFs per PF – 0
  • Number of DMA channels per PF – 1
Table 2.  DMA Channel Mapping w.r.t. MCDMA IP Parameters
PF MCDMA SW Channel User Channel
0 0 0
1 0 1
Level Two Title