Grid : BDII intergation

Created by Thomas Hartmann, last modified on Jan 27, 2022 14:34

While BDII has been deprecated in OSG, it's phase out in the WLCG/EGI seems to be slow as there are still dependencies on the information system like DIRAC used by LHCb, Belle 2 or ILC (DIRAC supports static end points, so a list of relevant information of a CondorCE should also be possible to be relayed to a DIRAC VO).

Prerequisites

  • a site level GIIS instance and/or a top level BDII
    • with the upcoming demise of the top level BDII at CERN please ensure, that your site level/top level BDII are queried by a BDII, that is known to your VOs, if they still depend on the information system
  • documentation on how to set up a GIIS/BDII seems to be somewhat aged - if you need to setup a fresh instance probably best would be to contact another site with a working instance

Setup

Create a configuration file for the information you want to publish towards BDII. The file goes to theCondor scheduler configuration (and not the CondorCE). The benchmark values should follow the up to date average for your cluster (probably).

Condor BDII integration : /etc/condor/config.d/...
> cat /etc/condor/config.d/90_40-ce-bdii.conf
# BDII Static Info and VOs
HTCONDORCE_VONames = atlas, cms, lhcb, dteam, ilc, belle, belle2
HTCONDORCE_SiteName = DESY-HH
HTCONDORCE_SPEC = [ hep_spec06 = 12.34] 
# GLUE2DomainID = DESY-HH
# HTCONDORCE_CORES = 8


Note, that it the information are in the GLUE2 schema, so it might not work with softwares still using GLUE1 like APEL.

Enable and run the bdii service, which is still System V, so keep fingers crossed, that systemd knows how to handle it.

systemctl enable bdii.service
systemctl start bdii.service

Your site level GIIS needs to know about the CondorCE and query the information from it. Add it with the correct URL to query from to its site-urls.conf.

BDII config - /etc/bdii/gip/site-urls.conf
> cat /etc/bdii/gip/site-urls.conf
...CONDORCE123 ldap://YOURCONDORCE.FQDN:2170/mds-vo-name=resource,o=grid

restart the GIIS/BDII service if necessary to pick up the new node.

Tests

If everything worked, you should be able to query your CondorCE through LDAP

ldapsearch -x -LLL -H ldap://YOURCONDORCE.FQDN:2170 -b 'o=glue'
ldapsearch -x -LLL -H ldap://YOURCONDORCE.FQDN:2170 -b 'o=grid'
ldapsearch -x -LLL -H ldap://YOURCONDORCE.FQDN:2170 -b 'o=infosys'

Similarly, your GIIS/BDII should query your CondorCE accordingly. If they do successfully, query them as well and check, if they include the information they queried from the CondorCE's bdii.


The script under

/var/lib/bdii/gip/provider/htcondor-ce-provider

knows about the relevant fields and prints them (if everything works correctly)

Notes

  • the BDII scripts are as of writing Python2 based and the services is in System V which might hamper deployment on more current OS releases as well as with HTCondorCE 5, that is aiming for full Python3 porting and CentOS 8 and similar as base platform


Attachments:

2019-09-26.htcondorce-bdii-apel.pdf (application/pdf)