Intel® FPGA SDK for OpenCL™ Pro Edition: Best Practices Guide

Download PDF
ID 683521
Date 3/28/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Introduction to Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide 2. Reviewing Your Kernel's report.html File 3. OpenCL Kernel Design Concepts 4. OpenCL Kernel Design Best Practices 5. Profiling Your Kernel to Identify Performance Bottlenecks 6. Strategies for Improving Single Work-Item Kernel Performance 7. Strategies for Improving NDRange Kernel Data Processing Efficiency 8. Strategies for Improving Memory Access Efficiency 9. Strategies for Optimizing FPGA Area Usage 10. Strategies for Optimizing Intel® Stratix® 10 OpenCL Designs 11. Strategies for Improving Performance in Your Host Application 12. Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide Archives A. Document Revision History for the Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide
1. Introduction to Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide x
1.1. FPGA Overview 1.2. Pipelines 1.3. Single Work-Item Kernel versus NDRange Kernel
2. Reviewing Your Kernel's report.html File x
2.1. High-Level Design Report Layout 2.2. Reviewing the Summary Report 2.3. Viewing Throughput Bottlenecks in the Design 2.4. Using Views 2.5. Analyzing Throughput 2.6. Reviewing Area Information 2.7. Optimizing an OpenCL Design Example Based on Information in the HTML Report 2.8. Accessing HLD FPGA Reports in JSON Format
2.4. Using Views x
2.4.1. Features of the System Viewer 2.4.2. Features of the Kernel Memory Viewer Kernel Memory List Kernel Memory Viewer Code View Details 2.4.3. Features of the Schedule Viewer
2.4.1. Features of the System Viewer x
2.4.1.1. Reviewing System Information 2.4.1.2. Reviewing Global Memory Information 2.4.1.3. Reviewing Block Information 2.4.1.4. Reviewing Cluster Information
2.5. Analyzing Throughput x
2.5.1. Reviewing Loop Information
2.6. Reviewing Area Information x
2.6.1. Area Report Message for Board Interface 2.6.2. Area Report Message for Function Overhead 2.6.3. Area Report Message for State 2.6.4. Area Report Message for Feedback 2.6.5. Area Report Messages for Private Variable Storage
2.6.5. Area Report Messages for Private Variable Storage x
2.6.5.1. Area Report Message for Constant Memory
3. OpenCL Kernel Design Concepts x
3.1. Kernels 3.2. Global Memory Interconnect 3.3. Local Memory 3.4. Loops in a Single Work-Item Kernel 3.5. Channels 3.6. Load-Store Units
3.3. Local Memory x
3.3.1. Changing the Memory Access Pattern Example
3.4. Loops in a Single Work-Item Kernel x
3.4.1. Trade-Off Between Initiation Interval and Maximum Frequency 3.4.2. Loop-Carried Dependencies that Affect the Initiation Interval of a Loop 3.4.3. Nested Loops 3.4.4. Loop Speculation 3.4.5. Loop Fusion 3.4.6. Loop Bottlenecks
3.4.3. Nested Loops x
3.4.3.1. Reducing the Area Consumed by Nested Loops Using loop_coalesce
3.6. Load-Store Units x
3.6.1. Load-Store Unit Types 3.6.2. Load-Store Unit Modifiers 3.6.3. Controlling the Load-Store Units 3.6.4. When to Use Each LSU
4. OpenCL Kernel Design Best Practices x
4.1. Transferring Data Via Intel® FPGA SDK for OpenCL™ Channels or OpenCL Pipes 4.2. Unrolling Loops 4.3. Optimizing Floating-Point Operations 4.4. Allocating Aligned Memory 4.5. Aligning a Struct with or without Padding 4.6. Maintaining Similar Structures for Vector Type Elements 4.7. Avoiding Pointer Aliasing 4.8. Avoid Expensive Functions 4.9. Avoiding Work-Item ID-Dependent Backward Branching
4.1. Transferring Data Via Intel® FPGA SDK for OpenCL™ Channels or OpenCL Pipes x
4.1.1. Characteristics of Channels and Pipes 4.1.2. Execution Order for Channels and Pipes 4.1.3. Optimizing Buffer Inference for Channels or Pipes 4.1.4. Best Practices for Channels and Pipes
4.3. Optimizing Floating-Point Operations x
4.3.1. Floating-Point versus Fixed-Point Representations
5. Profiling Your Kernel to Identify Performance Bottlenecks x
5.1. Best Practices for Profiling Your Kernel 5.2. Instrumenting the Kernel Pipeline with Performance Counters (-profile) 5.3. Obtaining Profiling Data During Runtime 5.4. Reducing Area Resource Use While Profiling 5.5. Temporal Performance Collection 5.6. Performance Data Types 5.7. Interpreting the Profiling Information 5.8. Profiler Analyses of Example OpenCL Design Scenarios 5.9. Intel® FPGA Dynamic Profiler for OpenCL™ Limitations
5.3. Obtaining Profiling Data During Runtime x
5.3.1. Invoking the Profiler Runtime Wrapper 5.3.2. Viewing Profiling Data Using Intel® VTune™ Profiler
5.3.1. Invoking the Profiler Runtime Wrapper x
5.3.1.1. Splitting Execution and Data Post Processing
5.5. Temporal Performance Collection x
5.5.1. Profiling Autorun Kernels
5.7. Interpreting the Profiling Information x
5.7.1. Stall, Occupancy, Bandwidth 5.7.2. Stalling Channels 5.7.3. Channel Depths
5.8. Profiler Analyses of Example OpenCL Design Scenarios x
5.8.1. High Stall Percentage 5.8.2. Low Occupancy Percentage 5.8.3. High Stall and High Occupancy Percentages 5.8.4. No Stalls, Low Occupancy Percentage, and Low Bandwidth 5.8.5. No Stalls, High Occupancy Percentage, and Low Bandwidth 5.8.6. High Stall and Low Occupancy Percentages
6. Strategies for Improving Single Work-Item Kernel Performance x
6.1. Addressing Single Work-Item Kernel Dependencies Based on Optimization Report Feedback 6.2. Good Design Practices for Single Work-Item Kernel
6.1. Addressing Single Work-Item Kernel Dependencies Based on Optimization Report Feedback x
6.1.1. Removing Loop-Carried Dependency 6.1.2. Relaxing Loop-Carried Dependency 6.1.3. Transferring Loop-Carried Dependency to Local Memory 6.1.4. Relaxing Loop-Carried Dependency by Inferring Shift Registers 6.1.5. Removing Loop-Carried Dependencies Caused by Accesses to Memory Arrays
7. Strategies for Improving NDRange Kernel Data Processing Efficiency x
7.1. Specifying a Maximum Work-group Size or a Required Work-Group Size 7.2. Kernel Vectorization 7.3. Multiple Compute Units 7.4. Combination of Compute Unit Replication and Kernel SIMD Vectorization 7.5. Reviewing Kernel Properties and Loop Unroll Status in the HTML Report
7.3. Multiple Compute Units x
7.3.1. Compute Unit Replication versus Kernel SIMD Vectorization
8. Strategies for Improving Memory Access Efficiency x
8.1. General Guidelines on Optimizing Memory Accesses 8.2. Optimize Global Memory Accesses 8.3. Performing Kernel Computations Using Constant, Local or Private Memory 8.4. Improving Kernel Performance by Banking the Local Memory 8.5. Optimizing Accesses to Local Memory by Controlling the Memory Replication Factor 8.6. Minimizing the Memory Dependencies for Loop Pipelining 8.7. Static Memory Coalescing
8.2. Optimize Global Memory Accesses x
8.2.1. Contiguous Memory Accesses 8.2.2. Manual Partitioning of Global Memory 8.2.3. Optimizing for Two or More Banks of Global Memory
8.2.2. Manual Partitioning of Global Memory x
8.2.2.1. Heterogeneous Memory Buffers
8.3. Performing Kernel Computations Using Constant, Local or Private Memory x
8.3.1. Constant Cache Memory 8.3.2. Preloading Data to Local Memory 8.3.3. Storing Variables and Arrays in Private Memory
8.4. Improving Kernel Performance by Banking the Local Memory x
8.4.1. Optimizing the Geometric Configuration of Local Memory Banks Based on Array Index
9. Strategies for Optimizing FPGA Area Usage x
9.1. Compilation Considerations 9.2. Board Variant Selection Considerations 9.3. Memory Access Considerations 9.4. Arithmetic Operation Considerations 9.5. Data Type Selection Considerations
10. Strategies for Optimizing Intel® Stratix® 10 OpenCL Designs x
10.1. Reducing Channel Overhead 10.2. Optimizing Loop Control 10.3. Simplifying Memory Access to Local Memories 10.4. On-Chip Storage of Reused Data 10.5. Optimizing Data Path Control 10.6. Creating RTL Modules
10.1. Reducing Channel Overhead x
10.1.1. Reducing the Number of Kernels 10.1.2. Using a Single Kernel to Describe Systolic Arrays 10.1.3. Using Non-Blocking Channels
10.2. Optimizing Loop Control x
10.2.1. Simplifying Loop-Carried Dependencies in Intel® Stratix® 10 OpenCL Designs
10.6. Creating RTL Modules x
10.6.1. Reset Recommendations
11. Strategies for Improving Performance in Your Host Application x
11.1. Multi-Threaded Host Application 11.2. Utilizing Hardware Kernel Invocation Queue
11.2. Utilizing Hardware Kernel Invocation Queue x
11.2.1. Double Buffered Host Application Utilizing Kernel Invocation Queue
1. Introduction to Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide
2. Reviewing Your Kernel's report.html File
3. OpenCL Kernel Design Concepts
4. OpenCL Kernel Design Best Practices
5. Profiling Your Kernel to Identify Performance Bottlenecks
6. Strategies for Improving Single Work-Item Kernel Performance
7. Strategies for Improving NDRange Kernel Data Processing Efficiency
8. Strategies for Improving Memory Access Efficiency
9. Strategies for Optimizing FPGA Area Usage
10. Strategies for Optimizing Intel® Stratix® 10 OpenCL Designs
11. Strategies for Improving Performance in Your Host Application
12. Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide Archives
A. Document Revision History for the Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide

2.4.2. Features of the Kernel Memory Viewer

Data movement is a bottleneck in many algorithms. The Kernel Memory Viewer in the High Level Design Report (report.html) shows you how the Intel® FPGA SDK for OpenCL™ Offline Compiler interprets the data connections and synthesizes memory for your kernel. Use the Kernel Memory Viewer to help you identify data movement bottlenecks in your kernel design.

Some patterns in memory accesses can cause undesired arbitration in the load-store units (LSUs), which can affect the throughput performance of your kernel. Use the Kernel Memory Viewer to identify unwanted arbitration in the LSUs.

Access the Kernel Memory Viewer by clicking System Viewers > Kernel Memory Viewer.

The following image illustrates the layout of the Kernel Memory Viewer:

Figure 19. Kernel Memory Viewer Layout

The Kernel Memory Viewer has the following panes:

Kernel Memory List
Lists all memories present in your design. When you select a memory name, you can view its graphical representation in the Kernel Memory Viewer pane.
Kernel Memory Viewer
Shows a graphical representation of the memory system or memory bank selected in the Kernel Memory List pane.
Code View
Shows the source code file for which the reports are generated.
Details
Shows the details of the memory system or memory bank selected in the Kernel Memory List pane.

Kernel Memory List

The Kernel Memory List pane displays a hierarchy of kernels with memories synthesized (RAMs, ROMs, and registers) and optimized away in that kernel.

Figure 20. Features and Details of the Kernel Memory List Pane

The following table describes each numbered feature highlighted in the above image:

Table 1.  Kernel Memory List Pane Icons and Labels
  Icon or Label Name Description
1 Kernel name The list of memories in your kernel can be expanded or collapsed. Memories that do not belong to any kernel are shown under (Other).
2 RAM

A RAM is a memory that has at least one write to it. The name of the RAM memory is same as its name in your design.

When you select a memory name, you can view a logical representation of the RAM in the Kernel Memory Viewer pane. By default, only the first bank of the memory system is displayed.

To select banks that you want the Kernel Memory Viewer pane to display:

  1. Expand the memory name.
  2. Clear the memory name check box to collapse all memory banks in the view.
  3. Select the memory name check box to show all memory banks in the view.
3 ROM

A ROM is a memory that is read-only. The name of the ROM memory is same as its name in your design.

When you select a memory name, you can view a logical representation of the ROM in the Kernel Memory Viewer pane. By default, only the first bank of the memory system is displayed.

To select banks that you want the Kernel Memory Viewer pane to display:

  1. Expand the memory name.
  2. Clear the memory name check box to collapse all memory banks in the view.
  3. Select the memory name check box to show all memory banks in the view.
4 Bank #num Bank

A memory bank is always associated with a RAM or a ROM. Each bank is named as Bank #num, where #num is the ID of the memory bank starting from 0.

  • Click the bank name to display the bank view in the Kernel Memory Viewer pane, which displays a graphical representation of the bank with all of its replicates and private copies. This view can help you focus on specific memory banks of a complex memory design.
  • Clear the memory bank name check box to collapse the bank in the logical representation of the memory.
  • Select the memory bank name check box to display the bank in the logical representation of the memory.
5 Register

A register is a kernel variable that is carried through the pipeline in registers (rather than being stored in a RAM or ROM). The name of the register is same as its name in your design.

You can implement a register variable either exclusively in FFs or in a combination of FFs and RAM-based FIFOs.

6 text label Optimized Away

A kernel variable can be optimized away because it is unused in your design, or compiler optimizations have transformed all uses of the variable such that it is unnecessary. The name of the optimized away variable is same as its name in your design.

7 Filter

Use the Kernel Memory List filter to selectively view the list of RAMs, ROMs, registers, and optimized away variables in your design.

When you clear the check box associated with an item in the filter, you hide all occurrences of that kind of item in the Kernel Memory List. Filter your Kernel Memory List to help you focus on a specific type of memory in your design.

Kernel Memory Viewer

In the Kernel Memory Viewer pane, you can view connections between loads and stores to specific logical ports on the banks in a memory system. You can also view the number of replicates and private copies created per bank for your memory system. You can see the following types of nodes in the Kernel Memory Viewer pane, depending on the kernel memory system and what you have selected in the Kernel Memory List pane:

Table 2.  Node Types Observed in the Kernel Memory Viewer Pane
Node Type Description
Memory node The memory system for a given variable in your design.
Bank node A bank in the memory system. A memory system contains at least one bank. A memory bank can connect to one or more port nodes. Only banks selected in the Kernel Memory List pane are shown.
Replication node A replication node shows memory bank replicates that are created to efficiently support multiple accesses to a local memory. A bank contains at least one replicate. You can view replicate nodes only when you view a memory bank by clicking its name in the Kernel Memory List pane.
Private-copy node A private-copy node shows private copies within a replicate that are created to allow simultaneous execution of multiple loop iterations. A replicate contains at least one private copy. You can view private-copy nodes only when you view a memory bank by clicking its name in the Kernel Memory List pane.
Port node

Each read or write access to a local memory is mapped to a port. The logical port for a bank. There are three types of port:

  • R: A read-only port
  • W: A write-only port
  • RW: A read and write port
LSU node A store (ST) or load (LD) node connected to the memory through port nodes.
Arbitration node An arbitration (ARB) node shows that LSUs compete for access to a shared port node, which can lead to stalls.
Port-sharing node A port-sharing node (SHARE) shows that LSUs have mutually exclusive access to a shared port node, so the load-store units are free from stalls.

Within the graphical representation of a memory in the Kernel Memory View pane, you can do the following:

  • Hover over any node to view the attributes of that node.
  • Hover over an LSU node to highlight the path from the LSU node to all ports that the LSU connects to.
  • Hover over a port node to highlight the path from the port node to all LSUs that read or write to the port node.
  • Click a node to select it and display the node attributes in the Details pane.

The following images illustrate examples of what you see in the Kernel Memory Viewer:

Figure 21. Logical Representation of a Memory in Kernel Memory Viewer Pane
Figure 22. Bank View of a Memory Bank in Kernel Memory Viewer Pane

Code View

The code view pane displays your source code. When you select a memory or a bank in the Kernel Memory Viewer pane, the code view pane highlights the line of your code where you declared the memory.

Details

The Details pane shows the attributes of the node selected in the Kernel Memory Viewer pane. For example, when you select a memory in a kernel, the Details pane displays information such as:
  • width and depths of the memory banks
  • memory layout
  • address-bit mapping
  • memory attributes that you specified in your source code

The content of the Details pane persists until you select a different node in the Kernel Memory Viewer pane.

Level Two Title