What's Included in the Intel® Distribution for Python*

Essential packages optimized for high-performance numerical and scientific computing.

author-image

作者

Package and Environment Managers

Python* package and environment managers are essential tools for productivity. These allow for installation, updating, and removal of no longer needed packages from the Python environment. They also manage multiple Python environments customized by a user for multiple purposes or needs.

Package

Description

References

How to Get It

conda*

A popular package manager that comes with the Python installation. You do not need to install it separately.

conda Documentation

conda install conda -c intel

conda install conda -c main

GitHub*

Download Intel® Distribution for Python*

Download Intel® AI Analytics Toolkit

Access Intel® Developer Cloud

pip

A classic Python package manager that comes with the Python installation. You do not need to install it separately.

PIP Documentation

conda install pip -c intel

conda install pip -c main

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

"

Data Processing and Modeling

Data processing is an essential part of numerical and data science workflows, and is based on numerical and statistical algorithms. It includes data collection, ingestion, preprocessing, normalization, transformation, aggregation, and analysis. Data analysis serves a basis for modeling of phenomena.

Package

Description

References

How to Get It

numpy

A foundational math library for core operations on N-dimensional arrays, optimized for Intel CPUs through Intel® oneAPI Math Kernel Library (oneMKL). Optimizations include linear algebra, fast Fourier transform (FFT), universal functions, and random number generators (RNG).

NumPy Documentation

 

conda install numpy -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

scipy

A scientific library for numerical integration, interpolation, optimization, linear algebra, and statistics. The Intel-optimized version is based on oneMKL.

SciPy Documentation

 

conda install scipy -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

numexpr

A lazy evaluation of numerical expressions for NumPy that optimizes memory use and makes use of all available CPU cores. The Intel-optimized version is based on oneMKL.

NumExpr* Documentation

 

 

conda install -c intel scipy

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

mkl_service

Python interfaces to oneMKL service functions for controlling MKL behavior.

It is not typically necessary to install this package with Intel optimized NumPy, SciPy, or NumExpr because they come with this package. However, it may be needed with other Python packages that rely on oneMKL if fine-grain control is necessary.

mkl_service Readme

 

conda install mkl-service -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

mkl_fft

Python interfaces to oneMKL FFT functions.

It is not typically necessary to install this package with Intel-optimized NumPy or SciPy because they come with this package. However, it may be needed to directly access MKL FFT interfaces rather than through NumPy or SciPy.

mkl_fft Readme

 

 

conda install mkl_fft -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

mkl_umath

Python interfaces to oneMKL vector math library (VML) functions.

It is not typically necessary to install this package with Intel-optimized NumPy or NumExpr because they come with this package. However, it may be needed to directly access MKL VML interfaces rather than through NumPy or NumExpr.

mkl_umath Readme

 

conda install mkl_umath -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

 

mkl_random

Python interfaces to oneMKL RNG functions. It is not typically necessary to install this package with Intel-optimized NumPy because it comes with this package. However, it may be needed to directly access MKL RNG interfaces rather than through NumPy.

mkl_random Readme

 

conda install mkl_random -c intel

Github

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

dpnp

The Data Parallel Extensions for NumPy library implements a subset of NumPy that can be offloaded to Intel GPUs.

Learn more about Data Parallel Extensions for Python.

dpnp Documentation

 

conda install dpnp -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

"

Foundational Machine Learning

Deep learning and machine learning are special models that allow a machine to automatically learn from data without programming it explicitly.

While Intel Distribution for Python includes a few foundational machine learning packages, Intel AI Analytics Toolkit provides a more comprehensive set of tools for deep learning and machine learning.

Package

Description

Reference

How to Get It

scikit-learn

A comprehensive set of machine learning algorithms for classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.

scikit-learn* Documentation

 

conda install scikit-learn -c intelpip install

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

scikit-learn-intelex

Intel® Extension for Scikit-learn* provides scikit-learn like interfaces for Intel® oneAPI Data Analytics Library (oneDAL).

It is not typically necessary to install Intel Extension for Scikit-learn as it comes with Intel optimized scikit-learn.

You can install the extension with standard scikit-learn to enable Intel optimizations and additional capabilities.

Scikit-learn Dcumentation

 

conda install scikit-learn-intelex -c intel

pip install scikit-learn-intelex

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

xgboost

XGBoost is a regularizing gradient boosting framework.

XGBoost Documentation

 

conda install xgboost -c intel

pip install xgboost

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

"

Python Interpreter and Compilers

The Python interpreter is the core of a versatile interactive experience with the language. It is expressive, intuitive, and fast enough for interactive usage. It is robust and secure for enterprise installations.

Python compilers are an important addition to the Python interpreter for scaling up Python applications. While initial prototyping work is mostly interactive, in a production setting the Python interpreter may become too slow to process large volumes of data.

Package

Description

Reference

How to Get It

python

A Python interpreter for best interactive experiences. Intel provides a robust and a secure build suitable for production enterprise environments.

Python Documentation

conda install python=[version] -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

numba

A just-in-time (JIT) compiler for a subset of Python and NumPy. It can generate highly efficient native code leveraging the latest Intel instructions such as Intel® Advanced Vector Extensions 2 (Intel® AVX2) and Intel® Advanced Vector Extensions 512 (Intel® AVX-512). It is also capable of generating multithreaded code through OpenMP* or oneAPI Threading Building Blocks (oneTBB) that uses all available CPU cores

Numba* Documentation

conda install numba -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

cython

An optimizing static compiler for Python, which is useful in writing C extensions for Python. Unlike Numba, which supports a subset of Python, Cython is a superset of the Python language.

 

Cython Documentation

conda install cython -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

 

"

Advanced Programming Packages

These are essential packages that enable fine-grained controls for data management, device management, concurrency, and parallelism.

Package

Description

Reference

How to Get It

tbb4py

A library of composable multithreading based on oneTBB. It enables composability of two or more multithreaded modules by using the oneTBB library as a common work scheduler.

oneTBB is a flexible performance library that supports scalable parallel programming using C++ code and is compliant with the ISO C++ standard.

oneTBB

conda install tbb4py -c intel

pip install tbb4py

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

 

smp

A library that controls thread affinity and static partitioning. Useful in static multiprocessing (SMP) programs where each process uses multithreaded library. Lack of affinity or partitioning controls provided by SMP leads to quadratic oversubscription of CPU cores by significantly degrading performance.

SMP Readme

conda install smp -c intel

pip install smp

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

mpi4py

A Python binding for MPI. Comes with Intel® MPI Library acceleration.

MPI for Python Documentation

 

conda install mpi4py -c intel

python -m pip install mpi4py

GitHub

Download Intel Distribution for Python

Intel AI Analytics Toolkit

Access Intel Developer Cloud

dpctl

A Data Parallel Control library that helps to manage CPU and GPU devices through Python. It also implements the Python Array API standard.

Learn more about Data Parallel Extensions for Python.

Data Parallel Control Documentation

conda install dpctl -c intel

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

"

Development Packages and Runtimes

Runtime packages are foundational for enabling Python performance. However, if you're a developer of a Python project that relies on built-in Intel libraries, you will also need the related development and runtime packages to ship your project.

Package

Description

Reference

How to Get It

mkl

A Python package that provides all you need to ship oneMKL with your project.

oneMKL Developer Guide

conda install mkl -c intel

pip install mkl

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

mkl-devel

A development package for oneMKL.

oneMKL Developer Guide

conda install mkl-devel -c intel

pip install mkl-devel

GitHub

mkl-dpcpp

A Python package that provides all you need to ship oneMKL with your project.

Learn more about Data Parallel Extensions for Python.

oneMKL Developer Guide

conda install mkl-dpcpp -c intel

pip install mkl-dpcpp

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

mkl-devel-dpcpp

A development package that provides all you need for oneMKL development for SYCL devices.

Learn more about Data Parallel Extensions for Python.

oneMKL Developer Guide

conda install mkl-devel-dpcpp -c intel

pip install mkl-devel-dpcpp

GitHub

daal

A Python package that provides all you need to ship oneDAL with your project.

oneDAL Documentation

conda install dal -c intel

pip install dal

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

dal-devel

A development package that provides all you need for development with oneDAL.

oneDAL Documentation

conda install -c intel dal-devel

pip install dal-devel

GitHub

ipp

Python package that provides all you need to ship Intel® Integrated Performance Primitives (Intel® IPP) with your project.

Intel IPP Developer Reference

conda install ipp -c intel

pip install ipp

ipp-devel

A development package that provides all you need for development with Intel IPP.

Intel IPP Developer Reference

conda install ipp-devel -c intel

pip install ipp-devel

tbb

A Python package that provides all you need to ship oneTBB with your project.

oneTBB Documentation

conda install tbb -c intel

pip install tbb

GitHub

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

tbb-devel

A development package that provides all you need for development with oneTBB.

oneTBB Documentation

conda install tbb-devel -c intel

pip install tbb-devel

GitHub

impi-rt

Python package that provides all you need to ship Intel® MPI with your project.

Intel MPI Library Developer Guide

conda install -c intel impi-rt

pip install impi-rt

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

intel-openmp

A Python package that provides all you need to ship OpenMP* with your project.

OpenMP Development Reference Guide

conda install intel-openmp -c intel

pip install intel-openmp

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

intel-opencl-rt

A Python package that provides all you need to ship OpenCL™ Runtimes with your project.

OpenCL Runtimes for Intel Processors

conda install intel-opencl-rt -c intel

pip install intel-opencl-rt

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

icc-rt

A Python package that provides all you need to ship your project that has native code compiled with Intel® C and Intel® C++ compilers.

Intel® C++ Compiler Classic Developer Guide and Reference

conda install icc_rt -c intel

pip install icc-rt

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

fortran-rt

A Python package that provides all you need to ship your project that has native code compiled with Intel® Fortran Compiler.

Intel Fortran Compiler Developer Guide and Reference

conda install fortran_rt -c intel

pip install fortran-rt

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

dpcpp-cpp-rt

A Python package that provides all you need to ship your project and that has native code compiled with Intel® oneAPI DPC++ Compiler.

Learn more about Data Parallel Extensions for Python.

 

Intel oneAPI DPC++/C++ Compiler Developer Guide and Reference

conda install dpcpp-cpp-rt -c intel

pip install dpcpp-cpp-rt

Download Intel Distribution for Python

Download Intel AI Analytics Toolkit

Access Intel Developer Cloud

 

"

Additional Recommended Packages

Intel Distribution for Python is an extensible platform that allows you to equip your numerical and data science workflow with all necessary tools and libraries. By using conda and PIP package managers, you can install additional packages not included in the stand-alone version.

Visualization and Interactive Environments

Python is ideal for interactive development. There are many Python tools and libraries that make interactive development easy and versatile. Following are a few essential tools and libraries:

Package

Description

bokeh

A library for interactive visualizations inside browsers.

jupyterlab

A web-based interactive development environment for Jupyter* Notebooks, code, and data. It supports a wide range of workflows in data science, scientific computing, and machine learning.

matplotlib

Plotting library with object-oriented API for embedding plots into applications.

plotly

Web-based library for out-of-the-box interactive data visualizations supporting multiple linked views, animation, and crosstalk integration.

pydot

Library for visualization of oriented and nonoriented graphs.

seaborn

Based on Matplotlib and serves a useful tool for visualizing statistical models, such as heatmaps and other visualizations that summarize data.

 

Data Processing and Modeling

Data processing is an essential part of numerical and data science workflows. It includes data collection, ingestion, preprocessing, normalization, transformation, aggregation, and analysis. Data processing is based on numerical and statistical algorithms. Data analysis serves a basis for modeling of phenomena.

Package

Description

pandas

A foundational library for operations on DataFrames.

modin

Accelerates pandas workflows and scales DataFrame operations across multiple nodes.

Intel® Distribution of Modin* integrates with OmniSci* for best performance.

sympy

A library for symbolic mathematics.

statsmodels

A library of essential statistical models, statistical tests, and statistical data exploration.

pyarrow

Python interfaces to Apache* Arrow, columnar data format optimized for in-memory DataFrame operations, query processing, and I/O (CSV, Parquet, and so on).

 

Image Processing and Computer Vision

Many scientific and AI applications work with images. Following are a few essential tools and libraries for image processing and computer vision.

Package

Description

scikit-image

Comprehensive library of image processing routines. The library is an image processing toolbox for SciPy.

opencv

Library targeting real-time computer vision problems.

 

Deep Learning

Deep learning and machine learning are special models, which allow a machine to automatically learn from a data without programming it explicitly.

Package

Description

tensorflow

TensorFlow* provides tutorials, examples, and other resources to speed up model building and create scalable machine learning solutions

pytorch

An open source machine learning framework that accelerates the path from research prototyping to production deployment

keras

Keras is an open source neural network library designed to provide fast experimentation with deep neural networks

 

Natural Language Processing

Find useful Python tools for processing and analyzing human language data.

Package

Description

nltk

A toolkit for natural language processing, which includes libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.

gensim

A library for unsupervised topic modeling and natural language processing that uses advanced statistical machine learning.

 

Data Crawling

Data scraping is a method of ingesting big data from a web. Data is typically extracted either by using direct APIs or via a general-purpose web crawler.

Package

Description

scrapy

An open source framework for extracting the data you need from websites.

beautifulsoup

A library that makes it easy to scrape information from web pages.

 

Data Management, Device Management, Concurrency, Parallelism & Communication

These are essential packages that enable fine-grained controls.

Package

Description

trio

A Python library for async concurrency and parallel I/O.

curio

A library for concurrency based on coroutines. It provides many useful abstractions for tasks, sockets, files, locks, and queues.

requests

A library for HTTP programming in a pythonic way.

 

Package and Environment Managers

Python package and environment managers are essential tools for productivity. These tools allow you to install additional, update existing, and remove no longer needed packages from a Python environment. They also manage multiple Python environments customized by a user for multiple purposes or needs.

Package

Description

mamba

An advanced implementation of conda optimized for performance. It allows parallel downloading of repository data and package files as well as much faster dependency resolution.

 

"