Computing : gitlab as a docker registry

There is a detailed description how to use gitlab as a Docker registry at

Following the instructions will get you very close. To create a test-case on gitlab.desy.de you essentially need only a few ingredients:

a Dockerfile containing the Docker image build instructions
CI/CD integration to automatically build images
possibly an access token

CI/CD

The CI/CD integration file in the test-case looks like this

#image: gitlab/dind

stages:
  - build

build-crystfel:
  stage: build
  image: docker:19.03.13
  services:
    - docker:19.03.12-dind
  variables:
    IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
  script:
    - docker info
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG

Dockerfile

A Dockerfile can be a very simple "Hello World". I usually use crystfel as a test-case. A rather old and crude version looks like this:

Dockerfile

# The base image. If not present, will be loaded from Dockerhub automatically
FROM centos:7
LABEL maintainer="fs" 
LABEL version="0.7.0" 
LABEL description="Container for the CrystFEL"

ENV CINCL=/usr/local/crystfel/include
ENV CLIBD=/usr/local/crystfel/data
ENV CCP4_SCR=/tmp
ENV PATH=/usr/local/crystfel/bin:$PATH

RUN yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm -y 2>&1 | grep -v "already installed and latest version"
RUN yum install -y gcc gcc-gfortran libstdc++ GConf2 gtk2 libXtst curl hdf5 hdf5-devel libstdc++-devel libstdc++-static libgcc gtk2-devel libpng-devel ncurses-devel fftw-devel cairo-gobject-devel pango-devel gtk-doc gsl-devel bzip2 make cmake gcc gfortran autoconf automake tar unzip 2>&1 | grep -v "already installed and latest version"
RUN mkdir -p /usr/local/src /usr/local/crystfel/bin

#
#  Crystfel itself
#
RUN curl -L http://www.desy.de/~twhite/crystfel/crystfel-0.7.0.tar.gz  | gzip -dc | tar -xv -C /usr/local
WORKDIR /usr/local/crystfel-0.7.0
RUN ./configure --prefix=/usr/local/crystfel
RUN make && make install
RUN rm -rf /usr/local/crystfel-0.7.0

#
#  MOSFLM
#
# v7.2.2 has problems. revert back to mosflm 7.2.1!
RUN curl -s -L http://www.desy.de/~schluenz/crystfel/mosflm-7.2.1.tgz | gzip -dc | tar -xv -C /usr/local/crystfel/bin/
RUN ln -sf /usr/local/crystfel/bin/mosflm-linux-64-noX11 /usr/local/crystfel/bin/mosflm
RUN ln -sf /usr/local/crystfel/bin/mosflm-linux-64-noX11 /usr/local/crystfel/bin/ipmosflm

#
#  aux files
#
RUN curl -s -L http://www.desy.de/~schluenz/crystfel/default.def -o  /usr/local/crystfel/include/default.def
RUN curl -s -L http://www.desy.de/~schluenz/crystfel/environ.def -o  /usr/local/crystfel/include/environ.def
RUN curl -s -L http://www.desy.de/~schluenz/crystfel/clibd.tgz | gzip -dc | tar -xv -C /usr/local/crystfel/
#RUN curl -s -L https://nims.desy.de/extra/desy-asg/SL/7/x86_64/common/repodata/repomd.xml -o /usr/local/test.out
#
#  ssh [might not be needed in general]
#
#RUN yum install -y openssh-clients openssh-server && ssh-keygen -A && \
#    sed -i 's/required\(.*pam_loginuid\)/optional\1/' /etc/pam.d/sshd

#
#  Clean up
#
RUN rm -rf /usr/local/src
RUN yum remove \*-devel -y
WORKDIR /

Running the Docker image

On maxwell you should get a compute node (no docker on login nodes), and simply run your docker image. In my case that works like this:

dockerrun -it gitlab.desy.de:5555/frank.schluenzen/crystfel:master

To be able to do so, you must be authorized to access the docker registry, which can be done by creating an access token. More information can be found at https://docs.docker.com/engine/reference/commandline/login/. One caveat: you need to know the tag of the image. Docker will by default look for a tag latest and fail if it doesn't exist:

dockerrun -it gitlab.desy.de:5555/frank.schluenzen/crystfel
docker: Error response from daemon: manifest for gitlab.desy.de:5555/frank.schluenzen/crystfel:latest not found: manifest unknown: manifest unknown

You can get the existing tags from the gitlab web, or via command line, for example

curl -s --header "PRIVATE-TOKEN: $(cat ~/.ssh/gitlab.crystfel.token)" https://gitlab.desy.de/api/v4/projects/106/repository/tags | jq '.[].name'

will list available tags. jp is a small json parser, python will do as well. master will not show up in the list of tags (not being a tag), but seems always a valid choice.