cgroup setup

cgroup setup

• cgroup setup differs between Scientific Linux 6 (kernel 2.6*) and CentOS 7 (kernel 3.10*)
  • under SL6/SysV you need to install libcgconfig tools etc. and configure them
  • under EL7 systemd will handle cgroup resources itself
  • note: all CPU shares of slots/cgroups are relative within the share of the condor parent cgroup relative to the other cgroups
  • we set slots to be partitionable to allow single and multi core jobs to mix

EL7

(thanks to Brian for the help)

• Condor job cgroups can be put under the system resource (maybe dedicated resource unit may be possible??)

BASE_CGROUP = /system.slice/condor.service

• with it a job's cgroup resource slice should be in

/sys/fs/cgroup/{cpu,cpuacct/memory/...}/system.slice/condor.service/condor_var_lib_condor_execute_slot1_*\@WN.FQDN.HERE/

SL6

• libcgroup needs to be installed and services cgconfig and cgred enabled/started
• to enable cgroups configure a parent cgroup 'htcondor'
  •  in the Condor worker node config BASE_CGROUP containing all jobs later on as sub-cgroups
  • and the basic cgroup config (the htcondor group defintion might be placed in /etc/cgconfig.d/... instead)

> cat /etc/cgconfig.conf
mount {
    cpuset    = /cgroup/cpuset;
    cpu    = /cgroup/cpu;
    cpuacct    = /cgroup/cpuacct;
    memory    = /cgroup/memory;
    devices    = /cgroup/devices;
    freezer    = /cgroup/freezer;
    net_cls    = /cgroup/net_cls;
    blkio    = /cgroup/blkio;
}
group htcondor {
      cpu {}
      cpuacct {}
      memory {}
      freezer {
      blkio {}
}

• with it, job cgroup resource infos/limits should be available around

/cgroup/{cpu/cpuacct/memory}/htcondor/condor_var_lib_condor_execute_slot1_*\@WN.FQDN.HERE/