Accelerator Functional Unit Developer Guide: Intel FPGA Programmable Acceleration Card N3000 Variants

Download PDF
ID 683190
Date 7/15/2022
Public
Document Table of Contents
Document Table of Contents x
1. About This Document 2. Introduction 3. High Level Description 4. Creating an N3000 FPGA Design 5. Capturing Signals in AFU with Signal Tap 6. Document Revision History for the Accelerator Functional Unit Developer Guide: Intel FPGA Programmable Acceleration Card N3000 Variants
1. About This Document x
1.1. Acronym List
2. Introduction x
2.1. Base Knowledge and Skills Prerequisites
2.1. Base Knowledge and Skills Prerequisites x
2.1.1. Considerations
3. High Level Description x
3.1. Steps for Creating Your AFU 3.2. N3000 Block Diagram 3.3. Factory Image Description
3.2. N3000 Block Diagram x
3.2.1. In-Line Data Path 3.2.2. Supported Ethernet Network Configurations 3.2.3. Provided Files 3.2.4. Internal Interfaces
3.2.3. Provided Files x
3.2.3.1. Directory Structure
3.2.4. Internal Interfaces x
3.2.4.1. Core Cache Interface (CCI-P) 3.2.4.2. Ethernet Interface 3.2.4.3. Ethernet MAC 3.2.4.4. 40G – 25G Gearbox 3.2.4.5. External Memory Interfaces 3.2.4.6. Ethernet MAC Wrapper Register Access
3.2.4.1. Core Cache Interface (CCI-P) x
3.2.4.1.1. FPGA Internal Register Access
3.2.4.5. External Memory Interfaces x
3.2.4.5.1. DDR4A and DDR4B 3.2.4.5.2. DDR4C 3.2.4.5.3. QDR4 Interface
4. Creating an N3000 FPGA Design x
4.1. Create New Project Directory 4.2. Create Your AFU Design Files 4.3. Build with make 4.4. Check Timing 4.5. Loading Your AFU into the Intel FPGA PAC N3000
4.2. Create Your AFU Design Files x
4.2.1. ccip_std_afu.sv 4.2.2. AFU File 4.2.3. QSF File 4.2.4. SDC File
4.5. Loading Your AFU into the Intel FPGA PAC N3000 x
4.5.1. Loading Your FPGA Image with JTAG 4.5.2. AFU Clocks 4.5.3. Creating an AFU with High Level Synthesis (HLS)
4.5.1. Loading Your FPGA Image with JTAG x
4.5.1.1. Preparing Your N3000 for JTAG 4.5.1.2. Disabling PCIe* Automatic Error Reporting (AER) 4.5.1.3. Using JTAG to Load the Intel® Arria® 10 *.sof file 4.5.1.4. How to Rescan PCIe* Bus and Re-enable PCIe* AER
4.5.2. AFU Clocks x
4.5.2.1. Hello AFU Example (uClk_usr) 4.5.2.2. Hello AFU Example (pll)
4.5.3. Creating an AFU with High Level Synthesis (HLS) x
4.5.3.1. Setting Up a Shell for HLS Development Work 4.5.3.2. Using the Initial Shell Design as a Shell 4.5.3.3. Compiling the Design and Producing a new N3000 FPGA Bitstream 4.5.3.4. Verifying Timing Constraints are Satisfied
5. Capturing Signals in AFU with Signal Tap x
5.1. Adding Signal Tap to the Design 5.2. Loading FPGA Image 5.3. Set Up Connections 5.4. How to Exit from the Debug Session 5.5. Troubleshooting Remote Debug Connections
1. About This Document
2. Introduction
3. High Level Description
4. Creating an N3000 FPGA Design
5. Capturing Signals in AFU with Signal Tap
6. Document Revision History for the Accelerator Functional Unit Developer Guide: Intel FPGA Programmable Acceleration Card N3000 Variants

4.5.2.1. Hello AFU Example (uClk_usr)

The module hello_afu.sv is modified to instantiate the BBB_ccip_async module to provide a clock crossing for the CCI-P interface between pClk and uClk_usr domains. The modified code is shown below: 
import ccip_if_pkg::*;
module hello_afu_uClk_usr
   (
    input  pClk,    // Core clock. CCI interface is synchronous to this clock.
    input  pClk_reset,  // CCI interface ACTIVE HIGH reset.
	
	input uClk_usr, //312.5 MHz user clock

    // CCI-P signals
    input  t_if_ccip_Rx pClk_cp2af_sRxPort,
    output t_if_ccip_Tx pClk_af2cp_sTxPort
    );

	`define AFU_ACCEL_UUID 128'h850adcc2_6ceb_4b22_9722_d43375b61c66
    // The AFU must respond with its AFU ID in response to MMIO reads of
    // the CCI-P device feature header (DFH).  The AFU ID is a unique ID
    // for a given program.  Here we generated one with the "uuidgen"
    // program and stored it in the AFU's JSON file.  ASE and synthesis
    // setup scripts automatically invoke the OPAE afu_json_mgr script
    // to extract the UUID into afu_json_info.vh.
    logic [127:0] afu_id = `AFU_ACCEL_UUID;

    logic [63:0] scratch_reg;
	
	//uClk_usr domain CCIP signals
	t_if_ccip_Tx af2cp_sTxPort;
	t_if_ccip_Rx cp2af_sRxPort;
	
	ccip_async_shim ccip_async_shim (
				    .bb_softreset    (pClk_reset),
				    .bb_clk          (pClk),
				    .bb_tx           (pClk_af2cp_sTxPort),
				    .bb_rx           (pClk_cp2af_sRxPort),
				    .afu_softreset   (reset),
				    .afu_clk         (uClk_usr),
				    .afu_tx          (af2cp_sTxPort),
				    .afu_rx          (cp2af_sRxPort)
				    );
   

    // The c0 header is normally used for memory read responses.
    // The header must be interpreted as an MMIO response when
    // c0 mmmioRdValid or mmioWrValid is set.  In these cases the
    // c0 header is cast into a ReqMmioHdr.
    t_ccip_c0_ReqMmioHdr mmioHdr;
    assign mmioHdr = t_ccip_c0_ReqMmioHdr'(cp2af_sRxPort.c0.hdr);

    //
    // Receive MMIO writes
    //
    always_ff @(posedge uClk_usr)
    begin
        if (reset)
        begin
            scratch_reg <= '0;
        end
        else
        begin
            // set the registers on MMIO write request
            // these are user-defined AFU registers at offset 0x40.
            if (cp2af_sRxPort.c0.mmioWrValid == 1)
            begin
                case (mmioHdr.address)
                    16'h0020: scratch_reg <= cp2af_sRxPort.c0.data[63:0];
                endcase
            end
        end
    end

    //
    // Handle MMIO reads.
    //
    always_ff @(posedge uClk_usr)
    begin
        if (reset)
        begin
            af2cp_sTxPort.c1.hdr <= '0;
            af2cp_sTxPort.c1.valid <= '0;
            af2cp_sTxPort.c0.hdr <= '0;
            af2cp_sTxPort.c0.valid <= '0;
            af2cp_sTxPort.c2.hdr <= '0;
            af2cp_sTxPort.c2.mmioRdValid <= '0;
        end
        else
        begin
            // Clear read response flag in case there was a response last cycle.
            af2cp_sTxPort.c2.mmioRdValid <= 0;

            // serve MMIO read requests
            if (cp2af_sRxPort.c0.mmioRdValid == 1'b1)
            begin
                // Copy TID, which the host needs to map the response to the request
                af2cp_sTxPort.c2.hdr.tid <= mmioHdr.tid;

                // Post response
                af2cp_sTxPort.c2.mmioRdValid <= 1;

                case (mmioHdr.address)
                    // AFU header
                    16'h0000: af2cp_sTxPort.c2.data <= {
                        4'b0001, // Feature type = AFU
                        8'b0,    // reserved
                        4'b0,    // afu minor revision = 0
                        7'b0,    // reserved
                        1'b1,    // end of DFH list = 1
                        24'b0,   // next DFH offset = 0
                        4'b0,    // afu major revision = 0
                        12'b0    // feature ID = 0
                        };

                    // AFU_ID_L
                    16'h0002: af2cp_sTxPort.c2.data <= afu_id[63:0];

                    // AFU_ID_H
                    16'h0004: af2cp_sTxPort.c2.data <= afu_id[127:64];

                    // DFH_RSVD0 and DFH_RSVD1
                    16'h0006: af2cp_sTxPort.c2.data <= 64'h0;
                    16'h0008: af2cp_sTxPort.c2.data <= 64'h0;

                    // Scratch Register.  Return the last value written
                    // to this MMIO address.
                    16'h0020: af2cp_sTxPort.c2.data <= scratch_reg;

                    default:  af2cp_sTxPort.c2.data <= 64'h0;
                endcase
            end
        end
    end
endmodule
You must edit ccip_std_afu.sv to connect the clock to your AFU. Addition to the ccip_std_afu.sv is shown below: 
hello_afu_pll afu
       (
        .pClk            (pClk),
        .pClk_reset          (pck_cp2af_softReset_T1),
		  . uClk_usr      (uClk_usr),

        .pClk_cp2af_sRxPort  (pck_cp2af_sRx_T1),
        .pClk_af2cp_sTxPort  (pck_af2cp_sTx_T0)
        );
The file afu.qsf is modified to source the ccip_async additions as shown below: 
# CCI-P async shim
source $AFU_SRC_ROOT/rtl/BBB_ccip_async/hw/par/ccip_async_addenda.qsf
The afu.sdc file has this additional constraint added: 
set_false_path -from [get_clocks {sys_csr_clk_pll|outclk[0]}]\
-to [get_clocks {fpga_top|inst_fiu_top|inst_ccip_fabric_top|inst_cvl_top|\
inst_user_clk|qph_user_clk_fpll_u0|xcvr_fpll_a10_0|outclk1}]
Then the build process is invoked with this make command: 
make 2x2x25G GUI=1 INCLUDE_DIAGNOSTICS=0 INCLUDE_MEMORY=0 \
PAC_VER_MAJOR=3 PAC_VER_MINOR=5 PAC_VER_PATCH=6 \
REVISION_ID=12345678 INCLUDE_AFU_PCIE1=0 USE_BBS_CLK=1
Level Two Title