Intel® oneAPI DPC++/C++ Compiler Developer Guide and Reference

Download PDF
ID 767253
Date 11/07/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Intel® oneAPI DPC++/C++ Compiler Developer Guide and Reference
Intel® oneAPI DPC++/C++ Compiler Developer Guide and Reference x
Intel® oneAPI DPC++/C++ Compiler Introduction Compiler Setup Compiler Reference Compilation Optimization and Programming Compatibility and Portability Notices and Disclaimers
Intel® oneAPI DPC++/C++ Compiler Introduction x
Get Help and Support Related Information
Compiler Setup x
Use the Command Line Use Eclipse Use Microsoft Visual Studio
Use the Command Line x
Specify Component Locations Invoke the Compiler Use the Command Line on Windows File Extensions Use Makefiles for Compilation Use CMake with the Compiler Use Compiler Options Specify Compiler Files Convert Projects to Use a Selected Compiler
Use Eclipse x
Add the Compiler to Eclipse Multiversion Compiler Support Use Cheat Sheets Create a Simple Eclipse Project Makefiles Use Intel Libraries with Eclipse
Use Microsoft Visual Studio x
Create a New Project Use the Intel® oneAPI DPC++/C++ Compiler Select the Compiler Version Specify a Base Platform Toolset Use Property Pages Use Intel® Libraries with Microsoft Visual Studio* Include MPI Support Dialog Box Help
Dialog Box Help x
Options: Compilers dialog box Use Intel® oneAPI DPC++/C++ Compiler dialog box Options: Intel Libraries for oneAPI dialog box
Compiler Reference x
C/C++/SYCL Calling Conventions Compiler Options Floating-Point Operations Attributes Intrinsics Libraries Macros Pragmas Error Handling
Compiler Options x
Alphabetical Option List General Rules for Compiler Options What Appears in the Compiler Option Descriptions Optimization Options Code Generation Options Interprocedural Optimization Options Advanced Optimization Options Profile Guided Optimization Options Optimization Report Options Offload Compilation, OpenMP*, and Parallel Processing Options Floating-Point Options Inlining Options Output, Debug, and Precompiled Header Options Preprocessor Options Component Control Options Language Options Data Options Compiler Diagnostic Options Compatibility Options Linking or Linker Options Miscellaneous Options Deprecated and Removed Compiler Options Display Option Information Alternate Compiler Options Portability and GCC-Compatible Warning Options
Optimization Options x
fast fbuiltin, Oi foptimize-sibling-calls GF nolib-inline O Od Ofast Os Ot Ox
Code Generation Options x
arch ax, Qax EH fasynchronous-unwind-tables fcf-protection, Qcf-protection fdata-sections, Gw fexceptions ffunction-sections, Gy fomit-frame-pointer Gd GR guard Gv m, Qm m64, Qm64 m80387 march masm mauto-arch, Qauto-arch mbranches-within-32B-boundaries, Qbranches-within-32B-boundaries mintrinsic-promote, Qintrinsic-promote momit-leaf-frame-pointer mtune, tune regcall, Qregcall x, Qx xHost, QxHost
Interprocedural Optimization Options x
flto ipo, Qipo
Advanced Optimization Options x
ffreestanding, Qfreestanding fjump-tables fvec-peel-loops, Qvec-peel-loops fvec-remainder-loops, Qvec-remainder-loops fvec-with-mask, Qvec-with-mask ipp-link, Qipp-link mno-gather, Qgather- mno-scatter, Qscatter- qactypes, Qactypes qdaal, Qdaal qipp, Qipp qmkl, Qmkl qmkl-ilp64, Qmkl-ilp64 qopt-assume-no-loop-carried-dep, Qopt-assume-no-loop-carried-dep qopt-dynamic-align, Qopt-dynamic-align qopt-for-throughput, Qopt-for-throughput qopt-mem-layout-trans, Qopt-mem-layout-trans qopt-multiple-gather-scatter-by-shuffles, Qopt-multiple-gather-scatter-by-shuffles qopt-prefetch, Qopt-prefetch qopt-streaming-stores, Qopt-streaming-stores qtbb, Qtbb unroll, Qunroll vec, Qvec vec-threshold, Qvec-threshold vecabi, Qvecabi
Profile Guided Optimization Options x
fprofile-dwo-dir fprofile-ml-use fprofile-sample-generate fprofile-sample-use
Optimization Report Options x
qopt-report, Qopt-report qopt-report-file, Qopt-report-file qopt-report-stdout, Qopt-report-stdout
Offload Compilation, OpenMP*, and Parallel Processing Options x
device-math-lib fintelfpga fiopenmp, Qiopenmp flink-huge-device-code fno-sycl-libspirv foffload-static-lib fopenmp fopenmp-declare-target-scalar-defaultmap, Qopenmp-declare-target-scalar-defaultmap fopenmp-device-code-split, Qopenmp-device-code-split fopenmp-device-lib fopenmp-max-parallel-link-jobs, Qopenmp-max-parallel-link-jobs fopenmp-target-buffers, Qopenmp-target-buffers fopenmp-targets, Qopenmp-targets fsycl fsycl-add-targets fsycl-dead-args-optimization fsycl-device-code-split fsycl-device-lib fsycl-device-obj fsycl-device-only fsycl-early-optimizations fsycl-enable-function-pointers fsycl-esimd-force-stateless-mem fsycl-explicit-simd fsycl-force-target fsycl-help fsycl-host-compiler fsycl-host-compiler-options fsycl-id-queries-fit-in-int fsycl-instrument-device-code fsycl-link fsycl-link-huge-device-code fsycl-link-targets fsycl-max-parallel-link-jobs fsycl-optimize-non-user-code fsycl-pstl-offload fsycl-rdc fsycl-targets fsycl-unnamed-lambda fsycl-use-bitcode ftarget-compile-fast ftarget-export-symbols nolibsycl qopenmp, Qopenmp qopenmp-link qopenmp-simd, Qopenmp-simd qopenmp-stubs, Qopenmp-stubs reuse-exe Wno-sycl-strict Xopenmp-target Xs Xsycl-target
Floating-Point Options x
ffp-contract fimf-absolute-error, Qimf-absolute-error fimf-accuracy-bits, Qimf-accuracy-bits fimf-arch-consistency, Qimf-arch-consistency fimf-domain-exclusion, Qimf-domain-exclusion fimf-max-error, Qimf-max-error fimf-precision, Qimf-precision fimf-use-svml, Qimf-use-svml fma, Qfma fp-model, fp fp-speculation, Qfp-speculation ftz, Qftz pc, Qpc
Inlining Options x
fgnu89-inline finline finline-functions
Output, Debug, and Precompiled Header Options x
c debug (Linux*) debug (Windows*) Fa fasm-blocks Fe Fo Fp fverbose-asm g gdwarf grecord-gcc-switches gsplit-dwarf o S use-msasm Y- Yc Yu Zi, Z7, ZI
Preprocessor Options x
B C D dD, QdD dM, QdM E EP FI H, QH I idirafter imacros iprefix iquote isystem iwithprefix iwithprefixbefore M, QM MD, QMD MF, QMF MG, QMG MM, QMM MMD, QMMD MQ, QMQ MT, QMT nostdinc++ P TP U undef X
Component Control Options x
Qoption
Language Options x
ansi fno-gnu-keywords fno-operator-names fno-rtti fpermissive fshort-enums fsyntax-only, Zs funsigned-char, J std, Qstd strict-ansi vd vmg x (type option) Zc Zg Zp
Data Options x
fcommon fkeep-static-consts, Qkeep-static-consts fmath-errno fpack-struct fpic fpie fstack-protector fstack-security-check fvisibility fzero-initialized-in-bss, Qzero-initialized-in-bss GA Gs GS mcmodel Qlong-double
Compiler Diagnostic Options x
w W Wabi Wall Wcheck-unicode-security Wcomment Wdeprecated Werror, WX Werror-all Wextra-tokens Wformat Wformat-security Wmain Wmissing-declarations Wmissing-prototypes Wpointer-arith Wreorder Wreturn-type Wshadow Wsign-compare Wstrict-aliasing Wstrict-prototypes Wtrigraphs Wuninitialized Wunknown-pragmas Wunused-function Wunused-variable Wwrite-strings
Compatibility Options x
gcc-toolchain vmv
Linking or Linker Options x
F (Windows*) fixed fortlib fuse-ld l L LD link MD MT nodefaultlibs no-intel-lib, Qno-intel-lib nostartfiles nostdlib pie pthread shared shared-intel shared-libgcc static static-intel static-libgcc static-libstdc++ T u (Linux*) v Wa Wl Wp Xlinker Zl
Miscellaneous Options x
dryrun dumpmachine dumpversion help nologo save-temps, Qsave-temps showIncludes sox, Qsox sysroot Tc TC Tp version
Floating-Point Operations x
Programming Tradeoffs in Floating-Point Applications Floating-Point Optimizations Denormal Numbers Floating-Point Environment Set the FTZ and DAZ Flags Tuning Performance IEEE Floating-Point Operations
Attributes x
align align_value allow_cpu_features const cpu_dispatch, cpu_specific target
Libraries x
Create Libraries Use Intel Shared Libraries on Linux Manage Libraries Redistribute Libraries When Deploying Applications Resolve References to Shared Libraries Sanitizers Intel's Memory Allocator Library SIMD Data Layout Templates Intel® C++ Class Libraries Intel's C++ Asynchronous I/O Extensions for Windows IEEE 754-2008 Binary Floating-Point Conformance Library Intel's Numeric String Conversion Library
SIMD Data Layout Templates x
Function Calls and Containers User-Level Interface Examples
Function Calls and Containers x
Construct an n_container Bounds
User-Level Interface x
SDLT Primitives soa1d_container aos1d_container n_container Bounds Accessors Proxy Objects Number Representation Indexes Convenience and Correctness
aos1d_container x
access_by
n_container x
Layouts Shape make_ n_container template function extent_d template function
Shape x
n_extent_generator
Bounds x
bounds_t sdlt::bounds Template Function n_bounds_t n_bounds_generator bounds_d Template Function
Accessors x
soa1d_container::accessor and aos1d_container::accessor soa1d_container::const_accessor and aos1d_container::const_accessor Accessor Concept
Proxy Objects x
Proxy ConstProxy
Number Representation x
aligned_offset fixed_offset
Indexes x
linear_index n_index_t n_index_generator index_d template function
Convenience and Correctness x
max_val min_val
Examples x
Efficiency with Structure of Arrays Example Complex SDLT Primitive Construction Example Forward Dependency Example Use of Offsets and Methods on a SDLT Primitive Example RGB to YUV Conversion Example
Intel® C++ Class Libraries x
C++ Classes and SIMD Operations Capabilities of C++ SIMD Classes Integer Vector Classes Floating-Point Vector Classes Classes Quick Reference Programming Example Intel's valarray Implementation
Integer Vector Classes x
Terms and Syntax Rules for Operators Assignment Operator Logical Operators Addition and Subtraction Operators Multiplication Operators Shift Operators Comparison Operators Conditional Select Operators Debug Operations Unpack Operators Pack Operators Clear MMX™ State Operator Integer Functions for Intel® Streaming SIMD Extensions Conversions between Fvec and Ivec
Floating-Point Vector Classes x
Fvec Syntax and Notation Data Alignment Conversions Constructors and Initialization Arithmetic Operators Minimum and Maximum Operators Logical Operators Compare Operators Conditional Select Operators for Fvec Classes Cacheability Support Operators Debug Operations Load and Store Operators Unpack Operators Move Mask Operators
Intel's C++ Asynchronous I/O Extensions for Windows x
Intel's C++ Asynchronous I/O Library for Windows Intel's C++ Asynchronous I/O Class for Windows
Intel's C++ Asynchronous I/O Library for Windows x
aio_read aio_write Example for aio_read and aio_write Functions aio_suspend Example for aio_suspend Function aio_error aio_return Example for aio_error and aio_return Functions aio_fsync aio_cancel Example for aio_cancel Function lio_listio Example for lio_listio Function Asynchronous I/O Function Errors
Intel's C++ Asynchronous I/O Class for Windows x
Template Class async_class get_last_operation_id wait get_status get_last_error get_error_operation_id stop_queue resume_queue clear_queue Example for Using async_class Template Class
IEEE 754-2008 Binary Floating-Point Conformance Library x
Intel® IEEE 754-2008 Binary Floating-Point Conformance Library and Usage Function List Homogeneous General-Computational Operations Functions General-Computational Operation Functions Quiet-Computational Operations Functions Signaling-Computational Operations Functions Non-Computational Operations Functions
Intel's Numeric String Conversion Library x
Use Intel's Numeric String Conversion Library Function List
Macros x
ISO Standard Predefined Macros Additional Predefined Macros Use Predefined Macros to Specify Intel® Compilers
Pragmas x
Intel-Specific Pragma Reference Intel-Supported Pragma Reference
Intel-Specific Pragma Reference x
block_loop/noblock_loop distribute_point inline, noinline, forceinline ivdep loop_count nofusion novector omp target variant dispatch ompx prefetch data prefetch/noprefetch unroll/nounroll unroll_and_jam/nounroll_and_jam vector
Compilation x
Compilation Overview Supported Environment Variables Pass Options to the Linker Specify Alternate Tools and Paths Use Configuration Files Use Response Files Global Symbols and Visibility Attributes for Linux* Save Compiler Information in Your Executable Link Debug Information Ahead of Time Compilation Device Offload Compilation Considerations Use a Third-Party Compiler as a Host Compiler for SYCL Code
Optimization and Programming x
Extensions OpenMP* Support Intel® oneAPI Level Zero Vectorization Instrumented Profile-Guided Optimization Hardware Profile-Guided Optimization Execution Frequency Feedback Execution Frequency and Branch Mispredict Feedback Notes on Windows Support Notes on the llvm-profgen and llvm-profdata tools High-Level Optimization Interprocedural Optimization Methods to Optimize Code Size Intel® oneAPI DPC++/C++ Compiler Math Library
OpenMP* Support x
Add OpenMP* Support Parallel Processing Model Worksharing Using OpenMP* Control Thread Allocation OpenMP* Pragmas OpenMP* Library Support OpenMP* Contexts OpenMP* Advanced Issues OpenMP* Implementation-Defined Behaviors OpenMP* Examples
OpenMP* Library Support x
OpenMP* Runtime Library Routines Intel® Compiler Extension Routines to OpenMP* OpenMP* Support Libraries Use the OpenMP Libraries Thread Affinity Interface OpenMP* Memory Spaces and Allocators
OpenMP* Contexts x
OpenMP* Context Selectors Score and Match Context Selectors
Intel® oneAPI Level Zero x
Intel® oneAPI Level Zero Switch Intel® oneAPI Level Zero Backend Specification Programming with the Intel® oneAPI Level Zero Backend
Vectorization x
Automatic Vectorization Explicit Vector Programming
Automatic Vectorization x
Vectorization Programming Guidelines Use Automatic Vectorization Vectorization and Loops Loop Constructs
Explicit Vector Programming x
User-Mandated or SIMD Vectorization SIMD-Enabled Functions SIMD-Enabled Function Pointers Function Annotations and the SIMD Directive for Vectorization Explicit SIMD SYCL Extension
Interprocedural Optimization x
Use Interprocedural Optimization Performance Considerations Create a Library from IPO Objects Inline Expansion of Functions
Intel® oneAPI DPC++/C++ Compiler Math Library x
Use the Intel® oneAPI DPC++/C++ Compiler Math Library Math Function List Trigonometric Functions Hyperbolic Functions Exponential Functions Special Functions Nearest Integer Functions Remainder Functions Miscellaneous Functions Complex Functions C99 Macros
Compatibility and Portability x
Standards Conformance GCC Compatibility and Interoperability Microsoft Compatibility Port from Microsoft Visual C++* to the Intel® oneAPI DPC++/C++ Compiler Port from GCC* to the Intel® oneAPI DPC++/C++ Compiler
Port from Microsoft Visual C++* to the Intel® oneAPI DPC++/C++ Compiler x
Modify Your makefile Other Considerations
Port from GCC* to the Intel® oneAPI DPC++/C++ Compiler x
Modify Your makefile Other Considerations
Intel® oneAPI DPC++/C++ Compiler Developer Guide and Reference

Hardware Profile-Guided Optimization

Hardware Profile-Guided Optimization (HWPGO) is an alternative to traditionally Instrumented Profile-Guided Optimization (IPGO).

Traditional IPGO requires a first compilation phase to generate a binary with instrumentation to track execution counts based on a training run.

With HWPGO, this instrumentation is not needed. Instead, the optimized binary's execution is sampled on Performance Monitoring Unit (PMU) events using a tool such as Linux perf or SEP, and a profile is generated from the PMU-based data and debug info.

A major benefit of HWPGO over IPGO is that the binary used for training can be highly optimized, and collection can occur in a production environment.

Another benefit is that the PMU can provide new types of hardware introspection not possible with software instrumentation. For example, the 2024.0 compiler has support for unpredictable branch profiles. The compiler can sometimes use such a profile to prefer Conditional Move (CMOV) to conditional branches.

Execution Frequency Feedback

  1. Compile with full optimization plus -fprofile-sample-generate.

    While HWPGO does not require instrumentation, -fprofile-sample-generate is recommended to ensure that useful debug information is generated.

    In this example, -fprofile-sample-generate is added to the application's existing optimization flags, -xCORE-AVX512 -Ofast: 

    icx -xCORE-AVX512 -Ofast -fprofile-sample-generate app.c -o app

    By default -fprofile-sample-generate does not affect optimizations and should not affect execution speed. Refer to -fprofile-sample-generate for more details.

    By default, debug info is embedded in object/executable files. To split debug info from those files, -fprofile-sample-generate with -gsplit-dwarf fprofile-dwo-dir=<dir> can be used together to specify where to store split .dwo files.

    On Windows, the lld linker must be used. The icx driver will ensure this when -fprofile-sample-generate is specified. Use lld-link /fprofile-sample-generate when invoking the linker directly.

  2. Create a PMU-based profile using SEP or Perf.

    Linux: 

    perf record -o app.perf.data -b -c 1000003 -e br_inst_retired.near_taken:uppp -- ./app

    Windows: 

    sep -start -out app.tb7 -ec BR_INST_RETIRED.NEAR_TAKEN:PRECISE=YES:SA=1000003:pdir:lbr:USR=YES -lbr no_filter:usr -perf-script ip,brstack -app .\app.exe

    NOTE:
    The sep tool only includes samples for the executable directly launched by -app in -perf-script output. This means, for example, that invoking app.exe via a wrapper script or batch file will not include app.exe samples. This will be improved in the future.

    The PMU-based profile data is now in app.perf.data or app.tb7, and on Windows, a partial textual representation is available as app.perf.data.script.

    The sampling period shown above (1000003) may need to be tuned depending on the application's characteristics and execution duration. The period chosen for each event type must be specified to llvm-profgen with the --sample-period option.

  3. Use the PMU profile to create an LLVM profile. The profile describes how frequently source-level code locations were observed executing.

    Linux: 

    llvm-profgen --perfdata app.perf.data --binary app --output app.freq.prof

    The process is the same on Windows, except you use the textual app.perf.data.script profile: 

    llvm-profgen --perfscript app.perf.data.script --binary app.exe --output app.freq.prof

  4. If steps 2-3 occurred multiple times, merge profiles with something like llvm-profdata merge --sample run1.freq.prof run2.freq.prof run3.freq.prof --output app.freq.prof. This is useful for training against multiple datasets.

  5. Recompile specifying the profile information to the compiler: 

    icx -xCORE-AVX512 -Ofast app.c -o app -fprofile-sample-use=app.freq.prof

    You may add -fprofile-sample-generate to the above if additional feedback iterations are desirable.

  6. Optionally, repeat by jumping back to step 2.

Execution Frequency and Branch Mispredict Feedback

  1. Compile with full optimization plus -fprofile-sample-generate: 

    icx -xCORE-AVX512 -Ofast -fprofile-sample-generate app.c -o app

  2. Create a PMU-based profile using SEP or Perf.

    The compiler can take advantage of both instruction execution and branch mispredict profiles. The two profiles can be collected simultaneously.

    Linux: 

    perf record -o app.perf.data -b -c 1000003 -e br_inst_retired.near_taken:uppp,br_misp_retired.all_branches:upp -- ./app

    Windows: 

    sep -start -out app.tb7 -ec BR_INST_RETIRED.NEAR_TAKEN:PRECISE=YES:SA=1000003:pdir:lbr:USR=YES,BR_MISP_RETIRED.ALL_BRANCHES:PRECISE=YES:SA=1000003:lbr:USR=YES -lbr no_filter:usr -perf-script event,ip,brstack -app .\app.exe

    The PMU-based profile data is now in app.perf.data or app.tb7, and on Windows, a partial textual representation is available as app.perf.data.script.

    NOTE:
    The additional event field requested of sep -- this event name field is required so that llvm-profgen can differentiate between PMU events.

  3. Use the single PMU profile to create two types of LLVM profiles. One will be the traditional execution frequency profile, and the other will be a profile of mispredicted branches.

    Linux: 

    llvm-profgen --perfdata app.perf.data --binary app --output app.freq.prof --sample-period 1000003 --perf-event br_inst_retired.near_taken:uppp 
llvm-profgen --perfdata app.perf.data --binary app --output app.misp.prof --sample-period 1000003 --perf-event mr_misp_retired.all_branches:upp --leading-ip-only

    The process is the same on Windows, except you will use SEP event names and the textual app.perf.data.script profile: 

    llvm-profgen --perfscript app.perf.data.script --binary app.exe --output app.freq.prof --sample-period 1000003 --perf-event BR_INST_RETIRED.NEAR_TAKEN:pdir
llvm-profgen --perfscript app.perf.data.script --binary app.exe --output app.misp.prof --sample-period 1000003 --perf-event BR_MISP_RETIRED.ALL_BRANCHES --leading-ip-only

    You should now have two source-level profiles: app.freq.prof and app.misp.prof.

  4. If steps 2-3 occurred multiple times, merge profiles with something like llvm-profdata merge --sample run1.freq.prof run2.freq.prof run3.freq.prof --output app.freq.prof. This is useful for training against multiple datasets.

    NOTE:
    The frequency and mispredict profiles should not be merged.

  5. Recompile specifying the profile information to the compiler. 

    icx -xCORE-AVX512 -Ofast app.c -o app -fprofile-sample-use=app.freq.prof -mllvm -unpredictable-hints-file=app.misp.prof

    You may add -fprofile-sample-generate to the above if additional feedback iterations are desirable.

  6. Optionally, repeat by jumping back to step 2.

Notes on Windows Support

The Intel® oneAPI DPC++/C++ Compiler provides an llvm-profgen tool to understand Common Object File Format (COFF) binaries with associated Debugging with Attributed Record Formats (DWARF) debug information. The -fprofile-sample-generate option ensures that this debug information is generated.

The Linux perf tool is unavailable on Windows, but Intel® VTune™ includes a sep tool that can perform the relevant Last Branch Records (LBR) sampling on hardware events on both Windows and Linux.

Notes on the llvm-profgen and llvm-profdata tools

To ensure that you use the versions of these tools corresponding to the product compiler, you may use the following to locate them: 

icx --print-prog-name=llvm-profgen

On Windows, icx is a command line-style driver, so you must use: 

icx /nologo /clang:--print-prog-name=llvm-profgen

Alternatively, the --include-intel-llvm option to setvars scripts will place these tools in PATH.

Parent topic: Optimization and Programming
Level Two Title