Intel® Distribution of OpenVINO™ Toolkit

753640
12/19/2024

Introduction

This package contains the Intel® Distribution of OpenVINO™ Toolkit software version 2024.6 for Linux*, Windows* and macOS*.

Available Downloads

  • Debian Linux*
  • Size: 29.8 MB
  • SHA256: 6AC2EB75715F1B40539F09E1426BCAFFA0BFF61F8F90464EFCDA2C5D9E6AB259
  • CentOS 7 (1908)*
  • Size: 52.3 MB
  • SHA256: CE7DF3BCC0437246E7C73A7C3BAB6ADE84477B7B7154657647EDE2179A15425A
  • Red Hat Enterprise Linux 8*
  • Size: 57.1 MB
  • SHA256: 12BFBE5F4F5D9B28C2469D6D123E246583173559CC8DB061A3016F5BDD95D6EE
  • Ubuntu 20.04 LTS*
  • Size: 48.9 MB
  • SHA256: 5B20F4810D1961AA72B80949A10FA50E78DFF81802828F75EB7540078B671E5A
  • Ubuntu 20.04 LTS*
  • Size: 32.9 MB
  • SHA256: E094E7C3E3A3931213AEEB6D6ABB6CC4D654F5CB4DF8DA2914158BA62E25E33A
  • Ubuntu 22.04 LTS*
  • Size: 51.4 MB
  • SHA256: E6D9D9D32E98FFE329C7DC3C1688BA0F9466308BE118117F8896049366AB39C0
  • Size: 52.6 MB
  • SHA256: D47281E02644D93FA299853F660DEC63B787F0A101AA393B4CDE71D1C8C00C18
  • macOS*
  • Size: 138.7 MB
  • SHA256: 62621BD51238820B2DF367DA047D96F9241F6F097EC3FE4F74D1A332499CE2D4
  • macOS*
  • Size: 33.4 MB
  • SHA256: 7780D1675C43DB511E42BA05EED93D5F1B33164CCD838ECF2C876EB283947907
  • Windows 11*, Windows 10*
  • Size: 108.4 MB
  • SHA256: 45A71A1E11F3E8A8109118E56434E79B3B2BFCC828B38B0E61BE55949F317A53

Detailed Description

What's New

  • OpenVINO™ 2024.6 release includes updates for enhanced stability and improved LLM performance.
  • Introduced support for Intel® Arc™ B-Series Graphics (formerly known as Battlemage)
  • Memory optimizations implemented to improve the inference time and LLM performance on NPUs.
  • Improved LLM performance with GenAI API optimizations

OpenVINO™ Runtime 


CPU Device Plugin 

  • KV cache now uses asymmetric 8-bit unsigned integer (U8) as the default precision, reducing memory stress for LLMs and increasing their performance. This option can be controlled by model meta data.
  • Quality and accuracy has been improved for selected models with several bug fixes. 

GPU Device Plugin 

  • Device memory copy optimizations have been introduced for inference with Intel® Arc™ B-Series Graphics (formerly known as Battlemage). Since it does not utilize L2 cache for copying memory between the device and host, a dedicated copy operation is used, if inputs or results are not expected in the device memory. ChatGLM4 inference on GPU has been optimized. 

NPU Device Plugin 

  • LLM performance and inference time has been improved with memory optimizations. 

OpenVINO.GenAI 

  • The encrypted_model_causal_lm sample is now available, showing how to decrypt a model. 

Installation instructions

You can choose how to install OpenVINO™ Runtime according to your operating system:

What's included in the download package

  • OpenVINO™ Runtime/Inference Engine for C/C++ and Python APIs

Helpful Links

NOTE: Links open in a new window.

This download is valid for the product(s) listed below.

Disclaimers1

Product and Performance Information

1

Intel is in the process of removing non-inclusive language from our current documentation, user interfaces, and code. Please note that retroactive changes are not always possible, and some non-inclusive language may remain in older documentation, user interfaces, and code.