Maxwell : rapids

The RAPIDS suite of open source software libraries and APIs gives you the ability to execute end-to-end data science and analytics pipelines entirely on GPUs. Licensed under Apache 2.0, RAPIDS is incubated by NVIDIA® based on extensive hardware and data science experience. RAPIDS utilizes NVIDIA CUDA® primitives for low-level compute optimization, and exposes GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.

Setup

a sample installation of rapids 22.04 is available as a conda environment. To activate the environment:

module load maxwell mamba/3.9
. mamba-init
mamba activate rapids-22.04
python -c 'import cudf' # works but of course only on GPU nodes

There is also an environment based on the "classical" conda installation, which you could use in case the mamba/3.9 python version doesn't suite you:

module load maxwell anaconda-python/3.8
. conda-init
conda activate rapids-22.04
python -c 'import cudf' # works but of course only on GPU nodes

Installation

the pre-installed environments might not match your required selection of python packages. In that case simply install it yourself (but not in your home-dir; it's too big). For example:

# https://rapids.ai/start.html#get-rapids provides recipes how to
module load maxwell mamba/3.9
. mamba-init
mamba create --prefix=/beegfs/desy/$USER/rapids/22.04 -c rapidsai -c nvidia -c conda-forge \
rapids=22.04 python=3.8 cudatoolkit=11.4 \
graphistry jupyterlab dash

mamba activate /beegfs/desy/$USER/rapids/22.04

# if the packages downloaded by conda exceed your quota you might need to change the location of pkgs, e.g.
conda config --add pkgs_dirs /beegfs/desy/$USER/.conda/pkgs
mkdir -p /beegfs/desy/$USER/.conda/pkgs