Skip to content

Latest commit

 

History

History
270 lines (174 loc) · 11.4 KB

INSTALL.md

File metadata and controls

270 lines (174 loc) · 11.4 KB
Raw

Installation from Sources

Required Software:

  • C/C++ Compiler

  • DPC++ Compiler and oneMKL if building with SYCL support

  • BLAS and LAPACK libraries - both provided by oneMKL

  • Python version 3.9 or higher

  • TBB library (repository contains script to download it)

  • Microsoft Visual Studio* (Windows* only)

  • MSYS2 (Windows* only)

  • make and dos2unix tools; install these packages using MSYS2 on Windows* as follows:

      pacman -S msys/make msys/dos2unix

For details, see System Requirements for oneDAL.

Note: the Intel(R) oneAPI components listed here can be installed together through the oneAPI Base Toolkit bundle:

https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html

All of these dependencies can alternatively be installed through the conda software, but doing so will require a few additional setup steps - see Conda Development Environment Setup for details.

Docker Development Environment

Docker file with the oneDAL development environment is available as an alternative to the manual setup.

Installation Steps

  1. Clone the sources from GitHub* as follows:

     git clone https://github.com/uxlfoundation/oneDAL.git

  2. Set the PATH environment variable to the MSYS2* bin directory (Windows* only). For example:

     set PATH=C:\msys64\usr\bin;%PATH%

  3. Set the environment variables for one of the supported C/C++ compilers, such as Intel(R)'s DPC compiler. For example:

    • Microsoft Visual Studio* 2022:

        call "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64

    • Intel(R) C++ Compiler 19.1 (Windows*):

        call "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\compilervars.bat" intel64

    • Intel(R) C++ Compiler 19.1 (Linux*):

        source /opt/intel/compilers_and_libraries/linux/bin/compilervars.sh intel64

    • Intel(R) oneAPI DPC++/C++ Compiler 2023.2 (Linux*):

        source /opt/intel/oneapi/compiler/latest/env/vars.sh

    • Intel(R) oneAPI DPC++/C++ Compiler 2023.2 (Windows*):

        call "C:\Program Files (x86)\Intel\oneAPI\compiler\latest\env\vars.bat"

    Note: if the Intel compilers were installed as part of a bundle such as oneAPI Base Toolkit, it's also possible to set the environment variables at once for all oneAPI components used here (compilers, MKL, oneMKL, TBB) through the more general script that they provide - for Linux:

         source /opt/intel/oneapi/setvars.sh

  4. Set up MKL:

    Note: if you used the general oneAPI setvars script from a Base Toolkit installation, this step will not be necessary as oneMKL will already have been set up.

    Download and install Intel(R) oneMKL. Set the environment variables for for Intel(R) oneMKL. For example:

    • Windows*:

        call "C:\Program Files (x86)\Intel\oneAPI\mkl\latest\env\vars.bat" intel64

    • Linux*:

        source /opt/intel/oneapi/mkl/latest/env/vars.sh

  5. Set up Intel(R) Threading Building Blocks (Intel(R) TBB):

    Note: if you used the general oneAPI setvars script from a Base Toolkit installation, this step will not be necessary as oneTBB will already have been set up.

    Download and install Intel(R) TBB. Set the environment variables for for Intel(R) TBB. For example:

    • oneTBB (Windows*):

        call "C:\Program Files (x86)\Intel\oneAPI\tbb\latest\env\vars.bat" intel64

    • oneTBB (Linux*):

        source /opt/intel/oneapi/tbb/latest/env/vars.sh intel64

    Alternatively, you can use scripts to do this for you (Linux*):

         ./dev/download_tbb.sh

  6. Download and install Python (version 3.9 or higher).

  7. Build oneDAL via command-line interface. Choose the appropriate commands based on the interface, platform, and the compiler you use. Interface and platform are required arguments of makefile while others are optional. Below you can find the set of examples for building oneDAL. You may use a combination of them to get the desired build configuration:

    • DAAL interfaces on Linux* using Intel(R) C++ Compiler:

        make -f makefile daal PLAT=lnx32e

    • DAAL interfaces on Linux* using GNU Compiler Collection*:

        make -f makefile daal PLAT=lnx32e COMPILER=gnu

    • oneAPI C++/DPC++ interfaces on Windows* using Intel(R) DPC++ compiler:

        make -f makefile oneapi PLAT=win32e

    • oneAPI C++ interfaces on Windows* using Microsoft Visual* C++ Compiler:

        make -f makefile oneapi_c PLAT=win32e COMPILER=vc

    • DAAL and oneAPI C++ interfaces on Linux* using GNU Compiler Collection*:

        make -f makefile daal oneapi_c PLAT=lnx32e COMPILER=gnu

It is possible to build oneDAL libraries with selected set of algorithms and/or CPU optimizations. CORE.ALGORITHMS.CUSTOM and REQCPUS makefile defines are used for it.

  • To build oneDAL with Linear Regression and Support Vector Machine algorithms, run:

          make -f makefile daal PLAT=win32e CORE.ALGORITHMS.CUSTOM="linear_regression svm" -j16

  • To build oneDAL with AVX2 and AVX512 CPU optimizations, run:

          make -f makefile daal PLAT=win32e REQCPU="avx2 avx512" -j16

  • To build oneDAL with Moments of Low Order algorithm and AVX2 CPU optimizations, run:

          make -f makefile daal PLAT=win32e CORE.ALGORITHMS.CUSTOM=low_order_moments REQCPU=avx2 -j16

On Linux* it is possible to build debug version of oneDAL or the version that allows to do kernel profiling using <ittnotify.h>.

  • To build debug version of oneDAL, run:

          make -f makefile daal oneapi_c PLAT=lnx32e REQDBG=yes

  • To build oneDAL with kernel profiling information, run:

          make -f makefile daal oneapi_c PLAT=lnx32e REQPROFILE=yes

NOTE: Built libraries are located in the __release_{os_name}[_{compiler_name}]/daal directory.

After having built the library, if one wishes to use it for building scikit-learn-intelex or for executing the usage examples, one can set the required environment variables to point to the generated build by sourcing the script that it creates under the env folder. The script will be located under __release_{os_name}[_{compiler_name}]/daal/latest/env/vars.sh and can be sourced with a POSIX-compliant shell such as bash, by executing something like the following from inside the __release* folder:

cd daal/latest
source env/vars.sh

The provided unit tests for the library can be executed through the Bazel system - see the Bazel docs for more information.

Examples of library usage will also be auto-generated as part of the build under path daal/latest/examples/daal/cpp/source. These can be built through CMake - assuming one starts from the release path __release_{os_name}[_{compiler_name}], the following would do:

cd daal/latest/examples/daal/cpp
mkdir -p build
cd build
cmake ..
make -j$(nproc)

This will generate executables under path daal/latest/examples/daal/cpp/_cmake_results/{platform_name}. They can be executed as follows (note that they require access to the data files under daal/latest/examples/daal/data), assuming that one starts from inside the build folder (as at the end of the previous step):

cd ..
./_cmake_results/{platform_name}/{example}

For example, in a Linux platform, assuming one wishes to execute the adaboost_dense_batch example:

./_cmake_results/intel_intel64_so/adaboost_dense_batch

Conda Development Environment Setup

The previous instructions assumed system-wide installs of the necessary dependencies. These can also be installed at a user-level through the conda or mamba ecosystems.

First, create a conda environment for building oneDAL, after conda has been installed:

conda create -y -n onedal_env
conda activate onedal_env

Then, install the necessary dependencies from the appropriate channels with conda:

  • Linux*:
conda install -y \
    -c https://software.repos.intel.com/python/conda/ \ `# Intel's repository`
    -c conda-forge \ `# conda-forge, for tools like 'make'`
    make python>=3.9 \ `# used by the build system`
    dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt \ `# Intel compiler packages`
    tbb tbb-devel \ `# required TBB packages`
    mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp \ `# required MKL packages`
    cmake `# required to build the examples only`
  • Windows*:
conda install -y^
    -c https://software.repos.intel.com/python/conda/^
    -c conda-forge^
    make dos2unix python>=3.9^
    dpcpp-cpp-rt dpcpp_win-64 intel-sycl-rt^
    tbb tbb-devel^
    mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp^
    cmake

Then modify the relevant environment variables to point to the conda-installed libraries:

  • Linux*:
export MKLROOT=${CONDA_PREFIX}
export TBBROOT=${CONDA_PREFIX}
export LD_LIBRARY_PATH="${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH}"
export LIBRARY_PATH="${CONDA_PREFIX}/lib:${LIBRARY_PATH}"
export CPATH="${CONDA_PREFIX}/include:${CPATH}"
export PATH="${CONDA_PREFIX}/bin:${PATH}"
export PKG_CONFIG_PATH="${CONDA_PREFIX}/lib/pkgconfig:${PKG_CONFIG_PATH}"
export CMAKE_PREFIX_PATH="${CONDA_PREFIX}/lib/cmake:${CMAKE_PREFIX_PATH}"
  • Windows*:
set MKLROOT=%CONDA_PREFIX%\Library
set TBBROOT=%CONDA_PREFIX%\Library
set "LD_LIBRARY_PATH=%CONDA_PREFIX%\Library\lib;%LD_LIBRARY_PATH%"
set "LIBRARY_PATH=%CONDA_PREFIX%\Library\lib;%LIBRARY_PATH%"
set "CPATH=%CONDA_PREFIX%\Library\include;%CPATH%"
set "PATH=%CONDA_PREFIX%\Library\bin;%PATH%"
set "PKG_CONFIG_PATH=%CONDA_PREFIX%\Library\lib\pkgconfig;%PKG_CONFIG_PATH%"
set "CMAKE_PREFIX_PATH=%CONDA_PREFIX%\Library\lib\cmake;%CMAKE_PREFIX_PATH%"

After that, it should be possible to build oneDAL and run the examples using the ICX compiler and the oneMKL libraries as per the instructions.