Introduction

We have a JupyterHub instance on https://naf-jhub.desy.de which is reachable from worldwide IP ranges, not just the DESY network.
In order to use the JupyterHub you need a NAF account, see Getting a NAF account for details.

Starting Jupyter

To start a Jupyter Job on the NAF JupyterHub, login with your DESY credentials and select Job options like your primary group and whether 
you wish to use a GPU slot.

You then need to wait until HTCondor starts your jupyter job and the notebook server is spawned. 
This takesabout 20, so don't be too impatient.

Once your server starts you see your AFS $HOME and can start a new Notebook by clicking on the "New" → "Notebook" → "Python 3"

ROOT Notebooks

We have ROOT installed on the Batch nodes. To use the Python bindings, simply

import ROOT

in a Python3 Notebook . Or Select the ROOT C++ Kernel directly from the dropdown menu.
For more information, see the official ROOT documentation https://root.cern.ch/notebooks/HowTos/HowTo_ROOT-Notebooks.html

JupyterLab User Extensions

If users want to use their own extensions they need to build jupyter lab in their HOME directory.
Execute the following command from a jupyter session terminal (new → Terminal)

jupyter lab build --app-dir=.local/share/jupyter/lab

Then selecting "JupyterLab with user extensions" on the Jupyter Spawn options page will launch jutyerlab from your 
$HOME directory and user extensions can be added.

Jupyter Kernel Environments

For normal Jupyter notebooks you can path on environment variables in the corresponding field in the options menu.

If you have build your own kernel for a JupyterLabs extension notebook, you can in addition hard-wire environment variables in the kernel's json definition. I.e., in

${HOME}/.local/share/jupyter/lab/kernels/YOURLABNAME/kernel.json

add a env dictionary object

{
...
,
"env": {"ROOTSYS":"/cvmfs/.../foo", "LIBPATH":"/cvmfs/.../baz",...}
}


.

Using Python Virtualenvs

It is recommended to use virtualenv for your environment.
To install a new virtualenv wiht the desired $NAME in your AFS home open a terminal ("New" → "Terminal") 
and run the following command:

python3 -m venv ~/$NAME

And to add the virtualenv to your jupyter notebook session first activate the virtualenv and install the ipython kernel

source ~/${NAME}/bin/activate

pip install ipykernel

ipython kernel install --user --name=${DISPLAY-NAME}

Where ${DISPLAY-NAME} is the text you want to appear in the jupyter notebook selection menu,
it does not have to be the same as <NAME>

You can now start jupyter notebooks with the virtualenv kernel

Error Handling

Our current implementation of the JupyterHub is not perfect, here are the errors 
we are aware of and what to do about them.

Quick & Dirty

If your notebook acts strange or does not start up possible, you can first try to get into a clean state. I.e., logout from all existing jupyter sessions and check with `jupyter_q` if you have notebook jobs active (`jupyter_rm JOBID` hard removes a notebook job from Condor).

Timout during Spawn

if the spawn of the server takes more than 120 seconds, meaning if HTCondor job doesn't start in 
this time, the job is cancelled and the JupyterHub returns an error.

This can have various reasons, most likely it's general disturbance in the BIRD service
and not directly related to the JupyterHub. If you encounter this kind of issue and there is no 
notification on the login page, send an email to unix@desy.de

Server didn't respond in 60 seconds

This is an error that occurs when the HTCondor job for the jupyter notebook get scheduled but 
when it starts the node does not have AFS and the job goes into hold  state.

At the moment the JupyterHub does not recognise this properly 
and thinks the jobs started and waits for the notebook to become reachable which never happens.

In the future we will implement a proper error handling. At the moment users should send a quick email to bird.service@desy.de
with the error message. We will then disable the node and fix the AFS issues.

Could not find local schedd Error

If you get an error "Could not find local schedd" you 
simply need to log out and log in again.

The error results from the login cookie being valid but the kerberos token 
having expired. A fresh login creates a new kerberos token and the error does not
appear any longer.

Referencing

Acknowledgments and References

If you find the Jupyter on NAF resources useful for your work, we would greatly appreciate to learn about publications, which have been substantially benefiting from the Grid Resources. Drop us a mail at TOWHOM@desy.de. An acknowledgement in your work would also be greatly appreciated, for example: "This research was realized in part on Jupyter notebooks on the National Analysis Facility computational resources operated at Deutsches Elektronen-Synchrotron (DESY), Hamburg, Germany"
A reference to a publication 'Interactive analysis notebooks on DESY batch resources' https://doi.org/10.1088/1742-6596/396/4/042026, J. Reppin et. al

BibTeX reference

@article{Reppin2021,
  url = {https://doi.org/10.1007/s41781-021-00058-y},
  doi = {10.1007/s41781-021-00058-y},
  year = {2021},
  month = jun,
  publisher = {Computing and Software for Big Science},
  volume = {5},
  number = {1},
  pages = {16},
  author = {Reppin, J., Beyer, C., Hartmann, T., Schluenzen, F., Flemming, M., Sternberger, S., Kemp, Y. },
  title = {Interactive analysis notebooks on DESY batch resources},
  journal = {Computing and Software for Big Science}
}