Introduction
Notice
This feature is in a pre-production state and only limited resources are available!
Please contact it-asap3@desy.de before you start to use this feature, comments and feedback are appreciated as well!
Online Resources with Commissioning are intended for commissioning of a beamline, to test new setups, optimize existing setups etc. together with online resources from Maxwell.
Do not abuse this feature for regular data acquisition!
Analog to regular beamtimes, online resources from Maxwell can be reserved for commissioning runs as well. These resources are only available during the commissioning run and allow access to the beamline filesystem from the Maxwell node.
User access is only possible with a specific user account via the use of SSH public keys.
The reserved resources can be used interactively and/or via sbatch SLURM jobs. For additional details about SLURM, have a look at the Maxwell space.
These resources are reserved and allocated via the startCommissioning script. See below on how to reserve and use the resources.
Resource Allocation via startCommissioning
Online resources can be allocated from the startCommissioning script. Additionally, it is possible to add options for resource allocation, e.g. nodes with GPUs or dedicated nodes for the beamline.
See startCommissioning --help for all available options:
% startCommissioning --help
usage: startCommissioning [-h] [--tag TAG] [--beamline BEAMLINE]
[--verbose] [--asapo] [--online]
[--feature FEATURE] [--resources PARTITION]
optional arguments:
-h, --help show this help message and exit
--tag TAG tag of your commissioning run, limited to 8 characters
--beamline BEAMLINE beamline that belongs to the commissioning run
--verbose more verbose output
--asapo Initialize ASAP::O
--online BETA: Reserve Maxwell Online Resources (without pre-
start)
--feature FEATURE BETA: Specifcy additional node features, only accepts
gpu
--resources PARTITION
BETA: Specify resource location, either common or
dedicated
Resource Selection
The flags --resources PARTITION and --feature FEATURE allow you some basic resource selection.
If none of the flags are specified, a single CPU node is taken from the common resources.
With --feature gpu, you will get a single node with a GPU (nVidia P100 as of 2021-08-16).
Some beamlines have acquired Maxwell resources for beamtimes, which allows them to use more resources. With --resources dedicated, these resources will be used.
The reservation in Slurm will use all available nodes from these dedicated resources.
Resource Allocation
Use the --online flag and optionally additional flags, like --feature or --resources:
# startCommissioning --tag my_test --beamline p00 --online
This will allocate resources on Maxwell for a commissioning run with the tag my_test on beamline p00.
Using Allocated Resources
After you have started your beamtime with Online Resources, first check the Commissioning Metadata File. The onlineAnalysis key contains all relevant information:
- Name of the reserved node(s)
- Name of the reservation, required to use for jobs with sbatch
- Beamtime specific account name
- Relative path to the SSH private key
To start using the resources, you must use the account specified in userAccount and the private key from sshPrivateKeyPath:
# ssh -l<userAccount> -i <sshPrivateKeyPath> <reservedNode>
The key is available on both, the beamline- and core filesystem. On the reserved node, the beamline filesystem is mounted on /beamline/<beamline>/commissioning
You should always read and write data from the core filesystem. Only write small results back to the beamline filesystem, e.g. if they are needed for changing the setup of the running commissioning run.