Contributor: Lin, Jie jie.lin@intel.com
Step 1 Overview
From Redis website https://redis.io/, "Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster."
Inter-procedural Optimization (IPO) that is used by Intel® C/C++ Compiler classic is an automatic, multi-step process that a static, topological analysis that span all of your source files are performed by the compiler to determine where you can benefit from specific optimizations. Now, the new Intel® DPC++/C++ Compiler which is LLVM-based uses Link Time Optimization (LTO) technology which is similar to the IPO concept. For more information on LLVM LTO, refer to
https://llvm.org/docs/LinkTimeOptimization.html.
While IPO is a static analysis of your program, Profile-Guided Optimizations (PGO) is a dynamic analysis that provides information to the compiler about areas of an application that are most frequently executed. Using PGO the compiler can be more selective and specific in optimizing the application. Intel® DPC++/C++ Compiler supports PGO by using profiles that can provide execution counts for instructions in the code and information on branches taken and function invocation. For more information, refer to Profile Guided Optimization https://clang.llvm.org/docs/UsersManual.html#id40
This article introduces how to build Redis with Intel® DPC++/C++ Compiler with both oridinary optimization options and advanced optimization options that uses IPO and PGO on Linux* platforms.
Step 2 Version Information
The instructions have been verified with Intel® oneAPI BaseToolkit 2021.4 on Ubuntu* 20.04.3 LTS (Focal Fossa).
GCC version information :
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.
Step 3 Downloading Redis Source Code
$ git clone https://github.com/redis/redis.git
- Prerequisites
Download and install Intel® oneAPI Base Toolkit
The download link could be found at https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit/download.html - Configuration
Set the Intel® DPC++/C++ Compiler environment variables by running the following commands assuming the default installation path of oneAPI is used:
$export PATH=/opt/intel/oneapi/compiler/latest/linux/bin:$PATH
$export LD_LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/linux/lib:/opt/intel/oneapi/compiler/latest/linux/compiler/lib/intel64_lin:$LD_LIBRARY_PATH
$export LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/linux/lib:$LIBRARY_PATH
Step 4 – Building Redis with Ordinary Optimization Options
Redis uses Makefile build system. So if you want to use Intel® DPC++/C++ Compiler, you can change the Makefile environment variables to reference driver names icx and icpx in Intel® DPC++/C++ Compiler. Type below command in the root directory of Redis source tree to enable compiler optimizations using AVX512 instruction set :
make CC=icx CXX=icpx CFLAGS=' -O3 -xCORE-AVX512 ' CXXFLAGS=' -O3 -xCORE-AVX512 '
Step 5 – Building Redis with IPO and PGO
In addtion to the ordinary optimization options of Intel® DPC++/C++ Compiler, we could further optimize Redis performance by enabling IPO and PGO.
- IPO
For IPO build, a bunch of LLVM binaries are used such as 'llvm-ar', 'llvm-ranlib' and 'llvm-nm'. Intel® oneAPI BaseToolkit 2021.4 does not provide 'llvm-nm'. So you must download it by yourself from https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.0/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-20HYPERLINK "https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.0/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-20.04.tar.xz".04.tar.xz and copy the 'llvm-nm' downloaded into your oneAPI installation directory /opt/intel/oneapi/compiler/latest/linux/bin (assuming you are using the default installation path of oneAPI).
Now for IPO build, you could type below command in the root directory of Redis source tree :
make CC=icx CXX=icpx CFLAGS=' -O3 -xHost -ipo ' CXXFLAGS=' -O3 -xHost -ipo ' LDFLAGS=' -ipo ' AR='llvm-ar' RANLIB='llvm-ranlib' NM='llvm-nm'
- PGO
Redis performance could be better if you also enable PGO for build. The PGO enabling requires below four phases :- The first phase is instrumentation by adding option '-fprofile-generate'.
make CC=icx CXX=icpx CFLAGS=' -O3 -xHost -fprofile-generate ' CXXFLAGS=' -O3 -xHost -fprofile-generate ' LDFLAGS=' -fprofile-generate '
- run the instumented binaries seperately to get profiling raw data (*.profraw files) .
For example, below two commands would get the profiling raw data files for 'redis-server' and 'redis-benchmark' separately.
LLVM_PROFILE_FILE="server-%m.profraw" ./redis/src/redis-server ./redis/src/redis.conf"
LLVM_PROFILE_FILE="benchmark-%m.profraw" ./redis/src/redis-benchmark -h 127.0.0.1 -p 6379 -n 10000000 -r 1000000 -c 50 -d 32 -t get,set,incr,lpush,lpop -P 64 -q
- create profiling data file with input raw *.profraw files. For example,
"llvm-profdata merge --output=benchmark.profdata benchmark*.profraw"
"llvm-profdata merge --output=server.profdata server*.profraw"
- Rebuild with the corresponding profiling data (*.profdata file). For example, below command builds with both IPO and the profiling data input with option '-fprofile-use' :
make CC=icx CXX=icpx CFLAGS=' -O3 -xHost -fprofile-use=./server.profdata -ipo ' CXXFLAGS=' -O3 -xHost -fprofile-use=./server.profdata -ipo ' LDFLAGS=' -fprofile-use=./server.profdata -ipo ' AR='llvm-ar' RANLIB='llvm-ranlib' NM='llvm-nm'
- The first phase is instrumentation by adding option '-fprofile-generate'.
Step 6 - Running Redis Benchmark
After build, Redis binaries will appear in the "src" sub-directory. Now you could try to run the benchmark following this link : https://redis.io/topics/benchmarks