Some python module won't work on Centos_7 anymore. In such cases you can create a conda environment for RedHat EL9 using singularity images, and run them either under Centos_7 or EL9.
Building a container from scratch
The best option is to create your own image, copy it to /beegfs or upload it to the container registry. A sample is available under /beegfs/desy/sys/singularity/micromamba-el9/. If you have a linux desktop you can create the singularity image directly on your desktop:
# some preparations on maxwell, only need to be done once ssh max-wgs # for example mk-beegfs # if you don't have a beegfs-folder yet mkdir -p /beegfs/desy/user/$USER/singularity /beegfs/desy/user/$USER/stmp /beegfs/desy/user/$USER/scache exit # on your desktop scp -r max-wgs:/beegfs/desy/sys/singularity/micromamba-el9/ . cd micromamba-el9 # modify, rename micromamba-el9.def to include the python modules you need for your project. # virtualGL is included in the image to be able running graphical applications on max-display using FastX singularity build --fakeroot micromamba-el9.sif micromamba-el9.def scp micromamba-el9.sif max-wgs:/beegfs/desy/user/$USER/singularity/ # back on maxwell ssh max-display # for example # particularly on share nodes like max-wgs, max-display /tmp or /scratch are quickly becoming too small. wont't hurt adding this to ~/.bashrc or ~/.zshrc export SINGULARITY_TMPDIR=/beegfs/desy/user/$USER/stmp export SINGULARITY_CACHEDIR=/beegfs/desy/user/$USER/scache # the home-directory is automatically imported into the singularity container. You might want to have other places included, for example export SARGS="-B /asap3:/asap3,/beegfs/desy/$USER:/beegfs/desy/$USER" # you don't want to include anything from ~/.local export PYTHONNOUSERSITE=1 # enter the image and run python singularity exec $SARGS /beegfs/desy/user/$USER/singularity/micromamba-el9.sif /opt/micromamba/bin/python Python 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> # the OS inside the container: singularity exec /beegfs/desy/user/$USER/singularity/micromamba-el9.sif bash Apptainer> cat /etc/os-release NAME="Rocky Linux" VERSION="9.3 (Blue Onyx)" ID="rocky" ID_LIKE="rhel centos fedora" VERSION_ID="9.3" PLATFORM_ID="platform:el9" PRETTY_NAME="Rocky Linux 9.3 (Blue Onyx)"
So you have your entire python environment inside the image. The filesystem only operates on a single file, and you haven't modified anything in your home-directory which presumably contains quite a bit of Centos_7 specific apps and code.
Building a singularity image directly on Maxwell will work exactly the same way, but requires some administrative preparation (subuid). Please let us know in case you'd like to build images on Maxwell.
Building singularity images directly on Maxwell
the above instructions with minimal changes:
# some preparations on maxwell, only need to be done once ssh max-wgs # for example mk-beegfs # if you don't have a beegfs-folder yet mkdir -p /beegfs/desy/user/$USER/singularity /beegfs/desy/user/$USER/stmp /beegfs/desy/user/$USER/scache cd /beegfs/desy/user/$USER/singularity cp -r max-wgs:/beegfs/desy/sys/singularity/micromamba-el9/ . cd micromamba-el9 # modify, rename micromamba-el9.def to include the python modules you need for your project. # virtualGL is included in the image to be able running graphical applications on max-display using FastX singularity build --fakeroot micromamba-el9.sif micromamba-el9.def cp micromamba-el9.sif max-wgs:/beegfs/desy/user/$USER/singularity/ # the home-directory is automatically imported into the singularity container. You might want to have other places included, for example export SARGS="-B /asap3:/asap3,/beegfs/desy/$USER:/beegfs/desy/$USER" # you don't want to include anything from ~/.local export PYTHONNOUSERSITE=1 # enter the image and run python singularity exec $SARGS /beegfs/desy/user/$USER/singularity/micromamba-el9.sif /opt/micromamba/bin/python Python 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> # the OS inside the container: singularity exec /beegfs/desy/user/$USER/singularity/micromamba-el9.sif bash Apptainer> cat /etc/os-release NAME="Rocky Linux" VERSION="9.3 (Blue Onyx)" ID="rocky" ID_LIKE="rhel centos fedora" VERSION_ID="9.3" PLATFORM_ID="platform:el9" PRETTY_NAME="Rocky Linux 9.3 (Blue Onyx)"
Using a pre-built image and create isolated conda-environments in your home directory
# Run the EL9 image, but redefine HOME. cp /beegfs/desy/sys/singularity/micromamba-el9/setup.sh ~/EL9/ # run the micromamba image with bash singularity exec --nv --home $HOME/EL9 /beegfs/desy/sys/singularity/micromamba-el9/micromamba-el9.sif bash source ~/setup.sh micromamba create --prefix=~/envs/test -c conda-forge python=3.10 micromamba activate ~/envs/test micromamba install -c conda-forge numpy # etc exit
Using the conda-environment on EL7 nodes
On EL7 nodes you will usually need to use the singularity image, though python itself will work. Make sure you don't have a mamba/conda setup predefined,
export PYTHONNOUSERSITE=1 # don't pull packages from ~/.local singularity exec --nv /beegfs/desy/sys/singularity/micromamba-el9/micromamba-el9.sif $HOME/EL9/envs/test/bin/python
Using the conda-environment on EL9 node
On EL9 nodes you can safely run python directly or through singularity:
export PYTHONNOUSERSITE=1 # don't pull packages from ~/.local # run directly $HOME/EL9/envs/test/bin/python # run through singularity singularity exec --nv /beegfs/desy/sys/singularity/micromamba-el9/micromamba-el9.sif $HOME/EL9/envs/test/bin/python