Computing : Building Containers (Docker, Apptainer/Singularity)

Singularity to Apptainer renaming

Singularity has been renamed to Apptainer in 2022 due to legal constraints. In general, just the name has changed and all options are the same. Replacing the command `singularity` with `apptainer` should work on updated systems.

Preliminary Remark

While a Apptainer/Singularity container can be run by an 'ordinary' user on Linux, building a fresh container requires sudo, so administrative rights to to a bit of magic during the build.
Obviously, sysadmins will not give users such rights on shared resources (wink)

To build and run a Docker container, a user has to be member of the local 'Docker' group. Unfortunately, the capabilities of this 'Docker' group are quite mighty comparable to sudo in some aspects - to quote the Docker documentation
"Only trusted users should be allowed to control your Docker daemon"
So, security minded sysadmins will be reluctant to grant such powers to users.

Recipes

Both, Docker and Apptainer/Singularity, containers are created from a template or recipe containing all the steps to setup the container (classically called Dockerfile for Docker and Apptainer/Singularity recipe for Apptainer/Singularity).

There is a wide variety on examples out there on how to write a Dockerfile or a Apptainer/Singularity recipes, so we will have only an intrudcution here

Dockerfile

A Dockerfile is somewhat "line oriented", Each line starting with a Docker command will result in it's own "layer", and all layers in the end combined and seen as a single container image
(technically, you have all these layers separately, which makes moving "a single image" around a bit cumbersome - but has the advantage that if you change only one line in the Dockerfile, only this section needs to be build again and the rest gets recycled)

cat Dockerfile

## from where do we bootstrap?
FROM centos:6
## always add comments and meta data so you know in two weeks, why you did things
MAINTAINER Thomas Hartmann @ DESY
## use RUN to do things in the container, e.g., create directories
RUN mkdir -p /etc/yum.repos.d/
## copy some files from the host namespace into the container
COPY host/etc/yum.repos.d/ /etc/yum.repos.d/
## and run additional commands in the container as updating the repository information
RUN yum -y update
## and each step bekomms another layer
RUN yum install -y curl
## so if you only change the previous one and keep all the rest, during a new build only the last step will be rerun

To create a new container from the Dockerfile run

(sudo) docker build -f /path/your/Dockerfile

Apptainer/Singularity Recipe

A recipe for a Apptainer/Singularity container is not much different from a Dockerfile except that it is not line/command oriented

cat Apptainer/Singularity

## generic things again: where do we bootstrap from, here we build from scratch
Bootstrap: yum
OSVersion: 6
MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/
Include: yum

## alternatively you can also start with a Docker container as starting point, e.g.,
# Bootstrap: docker
# From: centos:6

## meta data go in the %label section
%labels
Maintainer Thomas Hartmann @ DESY
Version 0.0.1

## copying files from the host namespace into the container happens in the %file section
%files
host/etc/yum.repos.d/* /etc/yum.repos.d/

## and all the things you want to run in the container go into the %post section
%post
mkdir -p /etc/yum.repos.d/
yum -y update
yum install -y curl

To create a container image "mycontainer.singularity" run

sudo singularity build mycontainer.singularity Singularity.file

This will walk through the whole recipe and create a single file "mycontainer.singularity", which you can copy around and can run as a user, e.g.,

singularity shell --options... mycontainer.singularity