Intel® Gaudi® Software Version 1.16.0

ID 标签 834275
已更新 12/23/2024
版本
公共

本文档可提供新的版本。客户应 单击此处 前往查看最新版本。

Optimize with Intel® Gaudi® AI Accelerators

  • Create new deep learning models or migrate existing code in minutes.

  • Deliver Generative AI performance with simplified development and increased productivity.

author-image

作者

We’re excited to introduce the release of Intel® Gaudi® software version 1.16.0, bringing forth numerous enhancements and updates for an improved GenAI development experience. We added preview support for vLLM, a fast and easy-to-use library for LLM inference and serving. Performance optimizations will be added in subsequent releases. For more information see the Intel Gaudi software vLLM.

Added preview support for Mixture of Experts (MoE) BF16 pretraining using Megatron-DeepSpeed. This release includes a recipe for pretraining Mixtral 4x7B and Mixtral 8x7B. For more information see Megatron-DeepSpeed models.

We have improved the performance of various LLM models, including LLaMA 2 7B/13B/70B FP8 for pre-training, and LLaMA 2 7B/70B FP8 and Mistral 7B for inference. For more information, check out the Intel Gaudi's model performance page.  

In this release, we’ve also upgraded the validated versions of several libraries, including PyTorch* 2.2.2, DeepSpeed 0.14.0, PyTorch Lightning 2.2.4, Megatron-DeepSpeed PR #372, and Ray 2.20. Lastly, a reminder that support for Debian* 10.10 will be deprecated following the 1.16 release. In subsequent releases, Lazy mode execution will be deprecated for Intel Gaudi software, and PyTorch Eager mode and torch.compile will be the default execution mode.

For more information on this release, see the release notes.