Computing : DESY specifics / Quickstart

This part is intended for users that do have general knowledg on how to use HTCondor and only need to know the DESY specific variables and setups to use BIRD successfully.

Data access

BIRD is relying heavily on mounted filesystems like AFS, DUST, PNFS/DCACHE, CVMFS etc. which means your data should be directly accessible on the workernodes already in 99% of the cases. If you doubt that please consider to check that on the workgroupserver and contact us rather than initiate heavy file transfers from insode your batchjob !

Accounting groups

The quotas in BIRD are depending on your accounting group that is similar to your primary group on the submitting server. If you need to change the accounting group make sure you choose a group that you are actually a member of and use the syntax below.

Default OS

Currently CentOS 7 (EL7) is default for all jobs running in BIRD. As the lifetime of Scientific Linux 6 (SL6) ended in Novemebr 2020, all computing nodes have been migrated to EL7.

If you need to run legacy application, that depend on SL6, please consider using Singularity containers. The NAF supports Singularity containers in the user context, so that you can start your jobs into self provided Singularity containers either from the Condor submission directly or inetractively from the command line as well.

Default/Lite jobs

If you do send a 'regular' job that does not specify any augmented runtime or multiple core usage you will get 1 core, 1.5/2 gb memory and 3 hours runtime after 3 hours the job will receive a SIGTERM (you can alter and trap the SIG by using the 'kill_sig = ' variable in your submit file) . If you manage to keep your job in these defaults you do benefit a lot from oversubscription as this type of jobs is able to run on any currently free resources in BIRD in contrast to all other jobs that are bound to your group quota.

If your job needs more resources like more memory or a longer runtime, you will have to specify these resources in your Condor job submission file.

Condor calculates an ongoing quota value for each user and group and weights the available resoruces accordingly to treat everybody fairly. So, requirering significantly more or larger resources will impact your quota negatively and your subsequent jobs might ave to wait longer for free resources.


DESY specific options for the submit file

+RequestRuntime = <seconds>  # request a runtime differing from the 3h default, up to 7 days are currently supported

RequestMemory = <MByte> # request any amount of memory different from the 1.5 gb default in 'MiB'

RequestDisk = <kByte> # request an amount of disk different from the 3GB default

RequestCpus = <num of cpus> # request a number of CPUs/cores different from the default (1 core)

+MyProject = "<project>" # request an accounting group different from your current primary group

Requirements = ( OpSysAndVer == "CentOS7" ) # explicitly requesting EL7 as operating system