we maintain installations of recent python versions on Maxwell. The installations reside in /software/python/<version> and can be configured using the module command:
module load maxwell module avail python --------------------------- /software/etc/modulefiles -------------------------------- python/3.10 python/3.6 python/3.7 python/3.9 # python/3.6 actually is system python 3.6! module load maxwell python/3.10 python --version Python 2.7.5 # ! python3 --version Python 3.10.5
A list of pre-installed packages can be found under Python modules on Maxwell. The packages were all installed using pip, and pip installations can quickly become inconsistent. So we usually try to avoid updates of the initial installation. If you have specific requirements and to avoid inconsistent package versions, you might want use virtual environment. Alternatively use mamba/conda which aims to keep package installations consistent.
Virtual environments
there are several python packages to manage virtual environments like venv, pyenv, pyenv-virtualenv. venv is very likely completely sufficient:
# create environment python3 -m venv /beegfs/desy/user/$USER/pyenvs/py39/tensorflow export PATH=/beegfs/desy/user/$USER/pyenvs/py39/tensorflow/bin:$PATH # activate environment . activate which python3 /beegfs/desy/user/schluenz/pyenvs/py39/tensorflow/bin/python3 python3 -m pip install tensorflow tensorboard matplotlib # during the installation pip will download packages and store them in ~/.cache/pip. You can safely remove caches after installation rm -rf ~/.cache/pip/ # the size of the installation in beegfs is about 1.5G