The Intel® oneAPI Video Processing Library (oneVPL) provides a single set of video-focused APIs for encoding, decoding, and video processing that supports current and future Intel integrated and discrete GPUs. The oneVPL APIs provide fast, high-quality video solutions by leveraging the latest hardware features of Intel GPUs.
Building oneVPL for Intel® GPUs
For building oneVPL for Intel® GPUs, first, we need to build the oneVPL Dispatcher from the oneVPL base repository. To use oneVPL for video processing you need to install at least one implementation. Here is a list of current implementations.
- oneVPL-intel-gpu for use on Intel Xe graphics and newer
- Media SDK for use on legacy (pre-Xe) Intel graphics
As shown in this diagram, the oneVPL Dispatcher dispatches the application to use either oneVPL GPU runtime, or the Media SDK GPU Runtime. We will support more GPU implementations in the future.
New Features in oneVPL API Functional Scope
Intel oneVPL API (2.x) has some key new features compared to the Media SDK API (1.35)
Improved session initialization with MFXLoad
MFXLoad replaces MFXInit/MFXInitEx in oneVPL as the entry point function for the oneVPL dispatcher. This enumerates and initializes all available GPU runtimes (libmfxhw64.so or libmfxgen.so). This upgrade to the initialization process means that runtimes can report capabilities so that the best implementation can be chosen based on the required codec and hardware.
Internal Memory Management
oneVPL API (>= API 2.0) introduces interface functions for allocating frames internally. The key advantage of using internal memory management over an external one is that the programmer does not have to take care of memory allocation for obtaining the number of working frame surfaces since the allocation is done by oneVPL. The surface returned by these oneVPL functions is a reference counted object and it is the programmer’s responsibility to call mfxFrameSurfaceInterface::Release after finishing all operations with the surface.
Note: Only the newer Intel GPUs (Gen12 or newer) support the internal Memory Management feature. Use of internal memory is not required for building oneVPL application. Developer can choose between internal and external memory management technique based on his requirement. If the developer is more focused on portability of the application, then external memory management is recommended. If the goal is to develop more performant application for newer Intel Hardware then internal memory is the most convenient option.
Building oneVPL Examples and Tools
Include Files
The oneVPL include folder is located at these locations on your development system.
$ONEAPI_ROOT/vpl/latest/include
Setting up the Environment
Run the following command to set up the necessary environment for oneVPL.
source <oneapi_install_dir>/setvars.sh
Prerequisite Software
- oneVPL
- CMake*
Build Program
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build .
Run Program
Following is the common format of the command line of any oneVPL application.
vpl_sample -i <InputFile> -o <OutputFile> -<other_parameters>
To see all the available options of a certain oneVPL sample application run the following command.
vpl_sample -?
oneVPL Examples
oneVPL examples provides rich code samples to show how to use the oneVPL API. The code samples are included in the examples directory of oneVPL base repository on GitHub*. There are many oneVPL examples available on this repository. However, we will only discuss ‘hello-*’ and ‘legacy-*’ examples here.
Dataset
There are various sample video files located in oneVPL/examples/content/ directory.
- cars_320x240.i420
- Video file format: YUV420
- Resolution: 320x240
- cars_320x240.nv12
- Video file format: NV12
- Resolution: 320x240
- cars_320x240.h265
- Video file format: hevc (Main), yuv420p(tv)
- Resolution: 320x240
- cars_320x240.mjpeg
- Video file format: mjpeg (Baseline), yuvj420p
- Resolution: 320x240
‘hello-*’ Examples
‘hello-*’ examples are the most basic examples of the oneVPL repository. These examples are located in hello directory of oneVPL base repository. The features/options of these examples are very limited. It is a good starting point for new oneVPL users. These examples use 2.x APIs of oneVPL which only works for newer GPUs (Xe or newer). Those who are interested in more advanced features can switch to sample_* tools. Here are some brief descriptions of the hello-* examples.
hello-createsession
This sample is a command line application that initializes a session. This is a hello world case to display oneVPL architecture.
Sample Command Line
hello-createsession -hw
Sample Output:
Session loaded: ApiVersion = 2.7 impl= Hardware:VAAPI
hello-decode
hello-decode is a minimal oneVPL decode application, that uses 2.2 or newer API with internal memory management. This sample decodes the H265 elementary video file to an output file (out.raw). The output file format is of raw native format (NV12 for GPU).
Sample Command Line:
./hello-decode -hw -i ../../../content/cars_320x240.h265
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Decoding ../../../content/cars_320x240.h265 -> out.raw
Output colorspace: NV12
Decoded 30 frames
hello-decvpp
The hello-decvpp example is an extension of hello-decode. Apart from decoding the H265 elementary stream to a raw format (dec_out.raw), this example also vpp the output with oneVPL, and writes the two vpp outputs to "vpp_640x480_out.raw" in raw native format and "vpp_128x96_out.raw" in raw BGRA format.
Sample Command Line:
./hello-decvpp -hw -i ../../../content/cars_320x240.h265
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Output colorspace: NV12
Decoding and VPP ../../../content/cars_320x240.h265 -> dec_out.raw and vpp_640x480_out.raw, vpp_128x96_out.raw
Decode and VPP processed 30 frames
hello-encode
hello-encode is a minimal oneVPL encode application, that uses 2.x API with internal memory management. This example takes a raw format (NV12 for GPU) video elementary stream as input and writes the output to an out.h265 file in H265 format after encoding it.
Sample Command Line:
./hello-encode -hw -i ../../../content/cars_320x240.i420 -w 320 -h 240
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Encoding ../../../content/cars_320x240.i420 -> out.h265
Input colorspace: NV12
Encoded 30 frames
hello-transcode
hello-transcode is a minimal oneVPL transcode application that uses oneVPL 2.2 API features including internal memory. This application takes a file containing a JPEG video elementary stream as an argument, decodes it, and encodes the output with oneVPL, and writes the encoded output to the file out.h265 in H.265 format.
Sample Command Line:
./hello-transcode -sw -i ../../../content/cars_320x240.mjpeg
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: SW
AccelerationMode via: NA
Path: /root/samples_test/oneVPL/lib/libvplswref64.so.1
Transcoding ../../../content/cars_320x240.mjpeg -> out.h265
Transcoded 30 frames
Note: Only Software/CPU implementation is available for hello-transcode
hello-vpp
hello-vpp is a minimal oneVPL VPP application that uses 2.x API with internal memory management. This sample is a command line application that takes a file containing a raw native format video elementary stream as an argument, processes it and writes the resized output to out.raw in BGRA raw video format.
Sample Command Line:
./hello-vpp -hw -i ../../../content/cars_320x240.i420 -w 320 -h 240
This command will write to the output file after resizing the frames.
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Processing ../../../content/cars_320x240.i420 -> out.raw
Processed 30 frames
‘legacy-*’ Examples
‘legacy-*’ examples are similar to ‘hello-*’ examples but ‘legacy-*’ uses the core API subset. ‘legacy-*’ examples are located in coreAPI directory of oneVPL base repository. These examples are universal which means they work both on older and newer Intel hardware and runtimes. These are the only functions and options that are common between MediaSDK runtime and the oneVPL runtime implementation. The advantages of programming with coreAPIs:
Increased Portability of the application from MediaSDK to oneVPL.
Can be compiled with both MediaSDK and oneVPL headers
Can be used for any Hardware supported by either Media SDK or oneVPL
legacy-decode
legacy-decode is a minimal oneVPL decode application using the core API subset. This sample is a command line application that takes a file containing an H.265 video elementary stream as an argument, decodes it with oneVPL, and writes the decoded output to the file out.raw in raw native format (NV12 for GPU). The functionality of the Legacy-decode is similar to the hello-decode in terms of functionality but it uses 1.x common APIs.
Sample Command Line:
./legacy-decode -hw -i ../../../content/cars_320x240.h265
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Decoding ../../../content/cars_320x240.h265 -> out.raw
Decoded 30 frames
legacy-encode
legacy-encode is a minimal oneVPL encode application that uses the core API subset. This sample is a command line application that takes a file containing a raw native format (NV12 for GPU) video elementary stream as an argument, encodes it with oneVPL and writes the encoded output to out.h265 in H.265 format. The functionality of the legacy-encode is similar to the hello-encode in terms of functionality but it uses 1.x common APIs.
Sample Command Line:
./legacy-encode -hw -i ../../../content/cars_320x240.i420 -w 320 -h 240
Sample Output:
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Encoding ../../../content/cars_320x240.i420 -> out.h265
Encoded 30 frames
legacy-vpp
legacy-vpp is a minimal oneVPL VPP application using the core API subset. This sample is a command line application that takes a file containing a raw native format video elementary stream as an argument, processes it with oneVPL writes the resized output to out.raw in BGRA raw video format. The functionality of the legacy-vpp is similar to the hello-vpp in terms of functionality but it uses 1.x common APIs.
Sample Command Line
./legacy-vpp -hw -i ../../../content/cars_320x240.i420 -w 320 -h 240
Sample Output
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Processing ../../../content/cars_320x240.i420 -> out.raw
Processed 30 frames
oneVPL Tools
There are oneVPL tools available in the oneVPL base repository which are a good starting point to get familiar with the oneVPL functionalities. The command line tools for checking available implementation supported by the current system are in the cli directory and the ‘sample-*’ tools are in the legacy directory.
Tools for Checking Implementation Capabilities
To check the implementation capabilities of oneVPL the following approaches can be used:,
First Approach:
vpl-inspect|grep Implementation
Sample output:
Implementation #0: mfx-gen
Implementation #1: oneAPI VPL CPU Implementation
ImplName: oneAPI VPL CPU Implementation
Second Approach (improved visibility in Linux environment):
system_analyzer
Sample output:
Implementation #0: mfx-gen
Library path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
AccelerationMode: MFX_ACCEL_MODE_VIA_VAAPI
ApiVersion: 2.7
Impl: MFX_IMPL_TYPE_HARDWARE
ImplName: mfx-gen
MediaAdapterType: MFX_MEDIA_INTEGRATED
VendorID: 0x8086
DeviceID: 0x9A49
GPU name: Intel® Iris® Xe Graphics GT2 (arch=Xe codename=Tiger Lake)
PCI BDF: 0000:00:02.00
DRMRenderNodeNum: 128
DeviceName: mfx-gen
Here, it is oneVPL dispatcher’s responsibility to load GPU runtimes (libmfxhw64.so or libmfxgen.so) depending on the system’s compatibility and selected codec. There are no plans for libmfxhw64.xo.1 to support new Gen (Gen12 or newer) platforms. Media SDK GPU runtime (libmfxhw64.so) runs on Gen9 and Gen11 while oneVPL GPU runtime (libmfxgen.so) supports Gen12, Xe, and newer hardware. Media SDK runtime supports API 1.35 and VPL runtime supports API 2.x (currently 2.8)
Loading GPU Runtimes
Figure 1 demonstrates the different steps of the GPU implementation selection process. In step 1, the dispatcher creates the loader to load oneVPL implementation. In step 2, the dispatcher filters out available implementations based on the selected hardware and codec. In the next step (step-3), oneVPL dispatcher walks through all the available runtime libraries (libmfx-gen, libvplswref, and future libraries) available to gather more details and returns a sorted list of oneVPL implementations which are associated with specific Hardware. For session creation (step 4), the best implementation for the current system, located at the 0th index of the sorted list, is loaded and initialized. After session creation is complete, an essential system handle is assigned to session that the loaded oneVPL library might use.
‘sample_*’ Tools
sample_* tools are the oneVPL tools with the greatest number of functionalities. The main advantages of using sample_encode over other encoders are:
- It supports almost all modern codecs available
- It supports different features starting from bitrate control to enabling different fixed functions which give users more control of the video decoding, encoding, and processing.
sample_encode
sample_encode is a oneVPL application that performs preprocessing and encoding of an uncompressed video stream of raw native format (NV12 for GPU) according to a specific video compression standard and can write the encoded video stream to a specified file.
Command Line format:
sample_encode h264|h265|mpeg2|mvc|jpeg -i InputYUVFile -o OutputEncodedFile -w width -h height -angle 180 -opencl
Sample Command Line:
sample_encode h265 -hw -i ../../../content/cars_320x240.nv12 -w 320 -h 240 -o output.h265
Sample Output:
Loaded Library configuration:
Version: 2.7
ImplName: mfx-gen
Adapter number : 0
Adapter type: integrated
DRMRenderNodeNum: 128
Used implementation number: 0
Processing started
Frame number: 30
Encoding fps: 994
Processing finished
sample_decode
sample_decode is an oneVPL decode application that takes a file containing an encoded video elementary stream as an argument, that performs decoding of various video compression formats, and writes the decoded output to the file in raw native format (NV12 for GPU).
Command Line format:
sample_decode h264|h265|mpeg2|mvc|jpeg -i in.bit -o out.yuv
Sample Command Line:
sample_decode h265 -i ../../../content/cars_320x240.h265 -o out.yuv
Sample Output:
Loaded Library configuration:
Version: 2.7
ImplName: mfx-gen
Adapter number : 0
Adapter type: integrated
DRMRenderNodeNum: 128
Used implementation number: 0
Decoding started
Frame number: 30, fps: 742.868, fread_fps: 0.000, fwrite_fps: 0.000358
Decoding finished
sample_vpp
sample_vpp is a oneVPL application that performs video processing of raw video sequences.
Command Line Format:
sample_vpp [Options] -i InputFile -o OutputFile
Sample Command Line:
sample_vpp -sw 320 -sh 240 -dw 320 -dh 240 -i out.raw -o crop.raw
Sample Output:
Loaded Library configuration:
Version: 2.7
ImplName: mfx-gen
Adapter number : 0
Adapter type: integrated
DRMRenderNodeNum: 128
Used implementation number: 0
VPP started
Frame number: 30
VPP finished
Total frames 30
Total time 0.01 sec
Frames per second 2151.000 fps
sample_multi_transcode
sample_multi_transcode performs the transcoding (decoding and encoding) of a video stream from one compressed video format to another, with optional video processing (resizing) of uncompressed video prior to encoding. The application supports multiple input and output streams meaning it can execute multiple transcoding sessions concurrently.
Command Line Format:
Format-1: sample_multi_transcode [options] [--] pipeline-description
Format-2: sample_multi_transcode [options] -par ParFile
ParFile is extension of what can be achieved by setting pipeline in the command line.
Sample Command Line:
sample_multi_transcode -hw -i::h265 ../../../content/cars_320x240.h265 -o::mpeg2 out.mpeg2
This command line transcodes a h265 video file to mpeg2 video format and writes it to out.mpeg2
.
Sample Output:
Session 0:
Loaded Library configuration:
Version: 2.7
ImplName: mfx-gen
Adapter number : 0
Adapter type: integrated
DRMRenderNodeNum: 128
Used implementation number: 0
Input video: HEVC
Output video: MPG2
Session 0 was NOT joined with other sessions
Transcoding started
Transcoding finished
Common transcoding time is 0.0288 sec
-------------------------------------------------------------------------------
*** session 0 [0x295d730] PASSED (MFX_ERR_NONE) 0.0287292 sec, 30 frames, 1044.235 fps
-hw -i::h265 ../../../content/cars_320x240.h265 -o::mpeg2 out.mpeg2