A Guide to Deploying AI Apps on AI PCs

Ramya Ravi, AI software marketing engineer, Intel | LinkedIn*
Eduardo Alvarez, senior AI solutions engineer, Intel | LinkedIn

Boost Your AI Skills Today

Looking to advance your expertise in AI? At the end of this article, make sure to review our resource collection.

What Is an AI PC?

AI PCs are the new generation of personal computers, which include a central processing unit (CPU), a graphic processing unit (GPU), and a neural processing unit (NPU) to provide power-efficient AI acceleration and handle AI tasks locally. Each processing unit has specific AI acceleration capabilities:

CPU: For AI tasks that require smaller workloads, sequential data, and low-latency inference
GPU: For larger workloads such as training deep neural networks that require parallel throughput
NPU: Dedicated hardware accelerator designed to handle AI workloads on your PC at low power for greater efficiency instead of processing data in the cloud

Benefits of Deployment on AI PCs

Power on Multiple Fronts: AI PCs are special because they pack multiple accelerators and a general compute engine on the same chip. This means you can tap into different architectures like the NPU for energy-efficient inference, the iGPU for more demanding tasks, and the CPU for traditional machine learning and complex operations. It's like having a unique tool for every situation.
Your Data, Your Rules: With AI PCs, you can run inference right on your device, so there's no need to send your data to third-party cloud services. This keeps your data secure and under your control, all within the comfort of your own premises.
No Internet? No Problem: AI PCs eliminate the need for a high-speed internet connection to perform meaningful AI tasks. By cutting out the middleman, your machine can handle AI workloads directly, no matter where you are.
Local Compute, Less Cloud Hassle: Since AI PCs run compute tasks locally, developers can deploy AI solutions that use their own device. This means less worrying about cloud infrastructure like load balancing and autoscaling, and more focus on building awesome applications.

Tips for AI PC Beginners

Optimizing an AI PC involves refining both hardware and software to deliver enhanced performance for AI tasks and efficient user experience. Here are a few key ways to optimize an AI PC.

Optimize Hardware Configurations

Use high-performance CPU, GPU, and NPU to improve AI processing speed.
Make sure to increase RAM (or have sufficient RAM) to handle large datasets and complex models.
Use fast storage solutions to reduce data loading times.

Software Optimizations

Use optimized AI frameworks like PyTorch* and TensorFlow* to take advantage of your specific hardware.
Use techniques like mixed-precision training, model pruning, and quantization to increase the computational speed and reduce memory use without losing accuracy.
Make use of AI model compilers like ONNX Runtime that are optimized for specific hardware.

System Maintenance

Avoid thermal throttling by keeping the PC clean and dust-free.
Track resource use to identify inefficiencies with the help of performance monitoring tools.

Deploy on Intel’s AI PC

Intel’s first AI PC platform features the Intel® Core™ Ultra processor. These processors can run over 500 optimized AI models on a single machine. Some of these AI models include Phi-2, Mistral, Llama, Bert, Whisper, and Stable Diffusion* 1.5, and can support different AI applications such as large language, diffusion, super resolution, object detection, and computer vision.

Check out our AI PC Notebooks GitHub Repository. There are various examples in the repository which demonstrates how to run different AI workloads on AI PCs. Each notebook has step-by-step instructions to set up your AI PC and implement the code samples.

Resource Library

Check out our latest documentation, video, and technical articles on deploying large language models with AI PCs and OpenVINO toolkit from Intel. This section is designed for developers of every skill level.

What you’ll learn:

How to generate music on your AI PC.
The advantages of optimizing LLMs using the OpenVINO™ toolkit.
How to use the OpenVINO toolkit to develop a GenAI assistant chatbot on an AI PC.

Get Started

Step 1: Optimize large language models using the OpenVINO toolkit.

This white paper explains the methods to optimize large language models through compression techniques and the advantages of using the OpenVINO toolkit for LLM deployment.

Step 2: Build a GenAI assistant chatbot on an AI PC.

Build a virtual travel assistant chatbot on an AI PC using the OpenVINO toolkit from Intel with this code sample and also watch this video to build a Large Language Model (LLM) assistant with Retrieval Augmented Generation (RAG) to query LLMs about external documents.