Transfer job executables
Depending on your workflow, you can either transfer the executable for each job to its batch node or let each job run the same executable from the shared storage with the switch
transfer_executable = True
in the job submission file.
Staging the executable for each job to their batch nodes has the advantage, that you can edit the executable afterwards without affecting submitted but not yet started jobs and that you are not depending on the shared file system.
Disadvantage might be, that job submission could be slower for a large executable, i.e., if the executable is a large binary blob
Overwrite variables in submission files during submission
If you have a hard-coded value in your submission file and want to modify it for the actual submission without editing the submission file, you can use the append feature - e.g.
> cat myjob.submit
...
+RequestRuntime = 10800
...condor_submit myjob.submit -append "+RequestJobRuntime = 14400"
Use environment variables in submit files
You can access environment variables in your submission file with the $ENV(YOURENVVAR) variable, e.g.,
export MYRUNTIME=10800> cat myjob.submit
...
+RequestRuntime=$ENV(MYRUNTIME)
...
Job-ID in File Names
To uniquely name files, you can make Condor put the job ID (or other ClassAds) into the path
E.g., the job ID is available in the ClassAd (Condor's internal key:value system)
ProcId
so that in the job submission file the output names could be tuned as
log = job_$(ProcId).log
output = job_$(ProcId).out
error = job_$(ProcId).err
Other probably interesting job ClassAds could be
- QDate : time the job was submitted in epoch
- ClusterId : the cluster you submitted your job. Maybe not so interesting, if you use just the NAF cluster - but could be handy to sort your files, if you also use other Condor clusters
- ExitCode : the exit code your job threw - maybe to sort logs on this one?
Low priority jobs for bottom fills
If your jobs can run with a low priority, i.e., if you have nor urgent needs for results but have a bulk of processing to do, consider to start your jobs with the flag
nice_user = True
enabled. This will mark your jobs with the lowest priority and your jobs will only be started on resources, if the Condor negotiator has no other users' jobs asking for resources. So that you can still make use of resources but not to annoy too much your colleagues in a hurry.
This is an unsupported feature at DESY as we do not run our jobs on desktops but on a regular dedicated batch system. These jobs will NOT run.