Computing : DEPRECATED: Mounting CERNBox/EOS locally

DEPRECATED: kept as legacy information

Please see IT: Utilities on how to mount DESYCloud/ Sync and Share/ Windows Homes or CERNBox over WebDAV

Note that these remote resources are not intended for running in your production and have to be mounted per host, i.e., don't use them for your batch jobs.




Question: How do I access my CERNBox from a batch job or how do I read/write data from/to EOS (and I don't want to use do copies of files over grid protocols (xropootd, gridFTP, https/WebDAV), i.e, it should look like the paths to the files are locally)?

Answer: You can (under CentOS7) mount directories on remote storage endpoints (e.g., CERN/EOS) over 'FUSE' so that the directory (you would normally look up via grid protocols) is 'put' into your local namespace.

Prerequisites:

  • a grid proxy at hand where you want to mount the remote storage end point
  • your environment is setup properly, so that you have gfal-tools from /cvmfs/grid.cern.ch around (see here)
  • CentOS7 (as under SL6 fuse/users is somewhat more fiddely)
  • know, that the I/O will be inefficient and slow - especially when touching many small files (git, compiling,...)
  • be careful what you do! you might easily screw up and delete files!

Grid: Mounting EOS ATLAS via xrootd as grid protocol

# check if you have a token ready
voms-proxy-info --all

# setup your environment
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/user/atlasLocalSetup.sh
... lsetup ...

#check, if you have setup everything correctly and that gfalFS is at hand
which gfalFS
  → /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/emi/1.0.0_v3.centos7.preview/usr/bin/gfalFS

# create in your (job's) home directory a mount point (and to be sure include in the path your user name to avoid clashes)
export MOUNTDIR="${TMPDIR}/${USER}/mnt/cerneos/xrootd"
gfalFS -s  ${MOUNTDIR} root://eosatlas.cern.ch:1094/eos/atlas/user/?/YOURUSERNAME

# the remote directory should be available now locally
ls ${MOUNTDIR}
→ a  b  c  d  e  f  g  h  i  j  k  l  m  n  o  p  q  r  s  t  u  v  w  x  y  z

# unmount the path at the end
# else the job cleanup might cause data losses or worse!

gfalFS_umount ${MOUNTDIR}

CERNBox

While there seems to be in principle a way yo speak to CERNBox/EOS via xrootd or WebDAV
  https://cern.service-now.com/service-portal/article.do?n=KB0003493
it is apparently not possible to authenticate with a grid proxy but only via username:password or a kerberos token at CERN.

However, the view on the directory returned from the EOS instance does not show any Users/ subdir or so but only infinite /eos/eos/ dubdirectories
If somebody manages a way to access CERNBox on its EOS end point, please drop a note...

If you feel adventurous

# get a Kerberos token to CERN
kinit -f -p -A YOURUSERNAME@CERN.CH
# lookup the end point
xrdfs  root://eosuser.cern.ch:1094/ ls

→ /eos
→ /lost+found
xrdfs  root://eosuser.cern.ch:1094/eos ls
→ /eos
→ /lost+found

# ad infinitum

Probably https/webDAVs is a better candidate for a working protocol than xrootd, but I had no look so far
https://cern.service-now.com/service-portal/view-request.do?n=RQF0928886