Intel core ultra processor badge

System Configuration Recommendations for AI PCs

AI PC Development

  • Transition projects from early AI development on the PC to cloud-based training to edge deployment

  • Intel® Core™ Ultra processor accelerates AI on the PC by combining CPU, GPU, and NPU.

  • Develop using OpenVINO™ Toolkit, Open Neural Network Exchange (ONNX), Web Neural Network API (WebNN) and more.

author-image

作者

Overview

To help AI developers configure a system that best meets their development needs, this article recommends:

  • Memory configurations
  • CPU and NPU models for Intel® Core™ Ultra processor
  • Intel integrated and discrete GPUs for different model classes, sizes, and bit precision levels

For example, if a developer wants to create a conversational agent or chatbot, they need to consider the size of the underlying LLM and the requisite memory or RAM size. The additional operational memory needed for development to avoid regular memory paging may cause unnecessary operational latencies.

 

Development Categories

Table 1 categorizes development based on the in-memory model size that is dependent on the number of parameters and the bit precision of the models used during development. Model quantization can be used in many instances to reduce memory requirements, system demand, and operational latency. If a developer is using a quantized model with a lower bit precision than natively defined for the model, the developer should reference the requirements associated with the appropriate bit precision.

Table 1. Development categories based on the size of the model in memory

Memory Configurations

Development System

Table 2 provides recommendations across several form factors for the memory configurations of the development system based on the model size categorization of the development tasks. The sizing recommendations take into account the approximate additional operational memory needs for the operating system on the development system. You may need to make additional considerations depending on the companion applications that are concurrently running on the system during the development process.

 

Table 2. AI model parameters mapped to recommended system configurations based on model size

Development Process

Table 3 provides similar recommendations across several form factors for progressively increasing performance levels as needed during development. These recommendations include augmenting the compute capability of the development system through one or more external GPUs (eGPU) or integrating one or more discrete GPUs on a desktop or workstation, which can be furnished via an Intel® Arc™ GPU or Intel® Arc™ Pro GPU.

Table 3. Recommended system configuration mapping (including CPU and GPU configurations) for various AI development performance levels