Overview
To help AI developers configure a system that best meets their development needs, this article recommends:
- Memory configurations
- CPU and NPU models for Intel® Core™ Ultra processor
- Intel integrated and discrete GPUs for different model classes, sizes, and bit precision levels
For example, if a developer wants to create a conversational agent or chatbot, they need to consider the size of the underlying LLM and the requisite memory or RAM size. The additional operational memory needed for development to avoid regular memory paging may cause unnecessary operational latencies.
Development Categories
Table 1 categorizes development based on the in-memory model size that is dependent on the number of parameters and the bit precision of the models used during development. Model quantization can be used in many instances to reduce memory requirements, system demand, and operational latency. If a developer is using a quantized model with a lower bit precision than natively defined for the model, the developer should reference the requirements associated with the appropriate bit precision.
Table 1. Development categories based on the size of the model in memory
Memory Configurations
Development System
Table 2 provides recommendations across several form factors for the memory configurations of the development system based on the model size categorization of the development tasks. The sizing recommendations take into account the approximate additional operational memory needs for the operating system on the development system. You may need to make additional considerations depending on the companion applications that are concurrently running on the system during the development process.
Table 2. AI model parameters mapped to recommended system configurations based on model size
Development Process
Table 3 provides similar recommendations across several form factors for progressively increasing performance levels as needed during development. These recommendations include augmenting the compute capability of the development system through one or more external GPUs (eGPU) or integrating one or more discrete GPUs on a desktop or workstation, which can be furnished via an Intel® Arc™ GPU or Intel® Arc™ Pro GPU.
Table 3. Recommended system configuration mapping (including CPU and GPU configurations) for various AI development performance levels
Processor Details
Table 4 provides additional information on the recommended Intel Core Ultra processors and Intel Arc GPUs. The recommended system models, SKUs, and parameters are provided for progressively increasing performance based on:
- Processor model
- Integrated GPU
- Need and specification of additional discrete and external GPUs that contribute to an improved AI developer experience
Table 4. Additional details on recommended system processor models and GPU configuration details for varying AI development performance levels
- p, e, and lpe refer to:
- p: Performance-cores
- e: Efficient-cores
- lpe: Low-power efficient cores optimized for low-power tasks