Intel Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual

Download PDF
ID 683193
Date 11/04/2019
Public
Document Table of Contents
Document Table of Contents x
1. Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual
1. Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual x
1.1. About this Document 1.2. Introduction 1.3. CCI-P Interface 1.4. AFU Requirements 1.5. Intel® FPGA Basic Building Blocks 1.6. Device Feature List 1.7. Document Revision History for Intel® Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual
1.1. About this Document x
1.1.1. Intended Audience 1.1.2. Conventions 1.1.3. Related Documentation 1.1.4. Acronym List for Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual 1.1.5. Acceleration Glossary
1.2. Introduction x
1.2.1. FPGA Interface Manager (FIM) 1.2.2. Intel® FPGA Interface Unit (FIU) 1.2.3. Memory and Cache Hierarchy
1.2.2. Intel® FPGA Interface Unit (FIU) x
1.2.2.1. FIU for Intel® FPGA PAC 1.2.2.2. FIU for Intel Integrated FPGA Platform 1.2.2.3. Comparison of FIU Capabilities
1.3. CCI-P Interface x
1.3.1. Signaling Information 1.3.2. Read and Write to Main Memory 1.3.3. Interrupts 1.3.4. UMsg 1.3.5. MMIO Accesses to I/O Memory 1.3.6. CCI-P Tx Signals 1.3.7. Tx Header Format 1.3.8. CCI-P Rx Signals 1.3.9. Multi-Cache Line Memory Requests 1.3.10. Byte Enable Memory Request ( Intel® FPGA PAC D5005) 1.3.11. Additional Control Signals 1.3.12. Protocol Flow 1.3.13. Ordering Rules 1.3.14. Timing Diagram 1.3.15. CCI-P Guidance
1.3.2. Read and Write to Main Memory x
1.3.2.1. Reading from Main Memory 1.3.2.2. Writing to Main Memory
1.3.8. CCI-P Rx Signals x
1.3.8.1. Rx Header and Rx Data Format
1.3.10. Byte Enable Memory Request ( Intel® FPGA PAC D5005) x
1.3.10.1. Mixing Byte Enable and Full Cache Line Accesses
1.3.12. Protocol Flow x
1.3.12.1. Upstream Requests 1.3.12.2. Downstream Requests
1.3.13. Ordering Rules x
1.3.13.1. Memory Requests 1.3.13.2. MMIO Requests
1.3.13.1. Memory Requests x
1.3.13.1.1. Memory Write Fence 1.3.13.1.2. Memory Consistency Explained
1.4. AFU Requirements x
1.4.1. Mandatory AFU CSR Definitions 1.4.2. AFU Discovery Flow 1.4.3. AFU_ID
1. Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual

1.2. Introduction

CCI-P is a host interface bus for an Accelerator Functional Unit (AFU) with separate header and data wires. It is intended for connecting an AFU to an FPGA Interface Unit (FIU) within an FPGA. This document defines the CCI-P protocol and signaling interface. It includes definitions for request types, header formats, timing diagrams and memory model.

In addition to the CCI-P signaling and protocol, this document also describes:
  1. Mandatory AFU registers required to design a CCI-P compliant AFU.
  2. Device Feature Lists (DFLs)—A standard for register organization that promotes modular design and easy enumeration of AFU features from the software.
  3. Intel® FPGA Basic Building Blocks (BBBs)—An architecture of defining reusable FPGA libraries that may consists of hardware and software modules.

The CCI-P offers an abstraction layer that can be implemented on top of a variety of platform interfaces like PCIe and UPI, thereby enabling interoperability of CCI-P compliant AFU across platforms.

The table below summarizes the features unique to the CCI-P interface for the AFU.

Table 5.   CCI-P Features
Feature Description
MMIO Request—CPU read/write to AFU I/O Memory
  • MMIO Read payload—4B, 8B
  • MMIO Write Payload—4B, 8B, 64B
    • MMIO writes could be combined by the x86 write combining buffer.
    • 64B MMIO writes require a CPU with capability of generation 64B Writes.
    • CPU for Integrated FPGA Platform can use AVX512 to generate 64B MMIO Write.
Memory Request
AFU read or write to memory.
  • Addressing Mode—Physical Addressing Mode
  • Addressing Width (CL aligned addresses)—42 bits (CL address)
  • Data Lengths—64 bytes (1 CL), 128 bytes (2 CLs), 256 bytes (4 CLs)
  • Byte Addressing—Not supported
FPGA Caching Hint (Integrated FPGA Platform only) The AFU can ask the FIU to cache the CL in a specific state. For requests directed to VL0, FIU attempts to cache the data in the requested state, given as a hint. Except for WrPush_I, cache hint requests on VH0 and VH1 are ignored.
Note: The caching hint is only a hint and provides no guarantee of final cache state. Ignoring a cache hint impacts performance but does not impact functionality.
  • <request>_I—No intention to cache
  • <request>_S—Desire to cache in shared (S) state
  • <request>_M—Desire to cache in modified (M) state
Virtual Channels (VC)

Physical links are presented to the AFU as virtual channels. The AFU can select the virtual channel for each memory request.

  • VL0—Low latency virtual channel (Mapped to UPI) (only for Integrated FPGA Platform).
  • VH0—High latency virtual channel. (Mapped to PCIe0). This virtual channel is tuned to handle large data transfers.
  • VH1—High latency virtual channel. (Mapped to PCIe1). This virtual channel is tuned to handle large data transfers (only for Integrated FPGA Platform).
  • Virtual Auto (VA)—FIU implements a policy optimized to achieve maximum cumulative bandwidth across all available physical links.
    • Latency—Expect to see high variance
    • Bandwidth—Expect to see high steady state bandwidth
UMsg (Integrated FPGA Platform only) Unordered notification directed from CPU to AFU
  • UMsgs data payload—64B
  • Number of UMsg supported—8 per AFU
Response Ordering Out of order responses
Upstream Requests Yes

Section Content
FPGA Interface Manager (FIM)
Intel FPGA Interface Unit (FIU)
Memory and Cache Hierarchy

Related Information
Level Two Title