This page is about a dCache service called "prometheus".  It has nothing to do with a similarly named monitoring solution.

What is dCache?

dCache is Free / Open-Source software for storing scientific data.  The software supports all parts of the scientific process: the data ingest from scientific instruments, any pre-processing steps, analysis, and data archival, with various options to support collaborations.  The software is developed and maintained by a consortium comprising of: DESY, Fermilab and NeIC, with contributions from others.  dCache is deployed by research institutes throughout the world and, through this deployment, supports many ground-breaking scientific developments. More information is available at the dCache website.

What is "prometheus"?

Prometheus is a self-contained instance of dCache, running on very modest hardware and using the latest version of dCache from the 'master' branch.

The dCache software project makes periodic "feature releases" (with reliable support periods).  These are based on whatever is in the 'master' branch at that release time.  Therefore, you can think of prometheus as providing a preview of what our next "feature release" of dCache will be able to do.

In order for prometheus to provide the latest 'master' version, it is upgraded daily.  This means the service is interrupted, at 06:00 CET/CEST, which results in the service being unavailable for a short period.

In addition to installing new software, the automated process also wipes all stored data and re-installs all reference data.  This is done deliberately, to ensure prometheus is used as intended, and the example data (some of which is modifiable) is available fresh each day.

For this reason, the update process may take some 20 minutes to complete.

Who should use dCache prometheus?

Prometheus is made available to give people an easy way to check compatibility with clients.  If you have a favourite client and want to check that it works with dCache, prometheus is a good way to go.

It's also a easy way to try out some of the cool, new features that dCache offers, even when those features are not yet part of an official dCache release.

Who should NOT use dCache prometheus?

The data stored on prometheus is wiped every day.  Therefore, the service is not intended for anyone who wants to store data for more than a few hours.

The hardware used to provide prometheus is limited.  It is not reasonable to attempt any performance-related tests or measurements: the results would be meaningless.

Similarly, the service does not provide much storage capacity.  Anyone who wishes to see how dCache scales handles large amounts of data will be disappointed with the available capacity.

The software is the latest cutting-edge and unreleased version of dCache.  Very occasionally, a problem slips past our QA process and causes trouble.  Therefore, prometheus is not an ideal choice if you need something reliable; e.g., for a live demo.

If prometheus is not what you need, you may still be able to take advantage of dCache's advance features without running your own service.  There are many research institutes that use dCache to provide a reliable, high-performance and fault-tolerant storage service.  You may find one such institute is already supporting your scientific community.

Since dCache is open-souce, you can download and run your own dCache instance.  Full details, including packages/containers and support, are available from the dCache website.

What's with these different certificates?

Prometheus has two PKI / X.509 server certificates it uses.  One is issued by an IGTF approved certificate authority (GermanGrid), the other is issued by a CAB-forum approved certificate authority (DFN-PKI).

The DFN-PKI is trusted by all major web-browsers.  Additionally, all mainstream operating systems have some way of deciding whether a certificate authority should be trusted, and most OSes trust DFN-PKI, so most "normal" clients will trust a certificate issued by DFN-PKI.  Unfortunately, IGTF certificates are typically not trusted by web-browsers or operating systems.

IGTF is the trust framework used within the "grid computing" community: most prominently by WLCG, but also in other areas of science.  Typically, grid clients will trust certificates issued by IGTF CAs and (almost always) do not trust those issued by CAB-forum CAs.

Due to technical limitations the the underlying network protocols (e.g,. TLS), dCache can only present one certificate when the client connects.  dCache can neither negotiate with the client nor present multiple server certificates.  Therefore, each endpoint (that uses X.509/PKI) must present either an IGTF certificate (from GermanGrid) or a CAB certificate (from DFN-PKI).

If you're using a "grid computing" client then use a IGTF endpoint.

If you're using regular software (e.g., a web browser) use a CAB endpoint.

If you've no idea and never heard of "grid": just go with the CAB endpoint – that's probably the right choice.

What protocols/endpoints does prometheus provide?

Prometheus provides the following endpoints:

ProtocolEndpointCertificateNotes
WebDAVhttp://prometheus.desy.de/-Insecure! No authentication (read-only access).
Simple HTML viewhttp://prometheus.desy.de/-Insecure! No authentication (read-only access). Only namespace exploration and file download supported.
WebDAVhttps://prometheus.desy.de/CAB
WebDAVhttps://prometheus.desy.de:2443/IGTF
Simple HTML viewhttps://prometheus.desy.de/CABOnly namespace exploration and file download supported.
dCacheViewhttps://prometheus.desy.de:3880/CABAdvanced JavaScript GUI
REST APIhttps://prometheus.desy.de:3880/api/v1/CAB
Swagger UIhttps://prometheus.desy.de:3880/api/v1/CAB
OpenAPI descriptionhttps://prometheus.desy.de:3880/api/v1/swagger.jsonCAB
gsiftp/GridFTPgsiftp://prometheus.desy.de/IGTFStandard port number: 2811.
FTPftp://prometheus.desy.de:22126/-Insecure! No authentication (read-only access).  Supports anonymous FTP.
FTPSftps://prometheus.desy.de/CABStandard port number is 21
xroot ("xrootd")xroot://prometheus.desy.de:1094/-Read-only access
xroot ("xrootd")xroot://prometheus.desy.de:1095/IGTFRequires GSI (X.509) authentication.
dcapdcap://prometheus.desy.de:22125/-Read-only access
dcapgsidcap://prometheus.desy.de:22128/IGTFRequires GSI (X.509) authentication.
SRMhttpg://prometheus.desy.de:8443/IGTFUses GSI (not TLS) for the handshake.
SRMhttps://prometheus.desy.de:8445/IGTFUses TLS
GridSitehttps://prometheus.desy.de:8445/IGTFGridSite delegation

What reference data is available?

Prometheus provides various test data to make testing dCache easier.  Most data has been configured so it is public (any dCache user and the anonymous user can read the data) but read-only.  There are some files and directories that are created with restricted access (as described below).  In addition, users have "home directories" in which they can modify content as they choose.

Unicode tests

The directory /UTF-8 contains various directories that use language-specific or less common characters.  These directories contain files that (similarly) contain less common characters.

Media data

The directories /Music and /Video contain audio and video files that are available under Creative Commons licenses.  These should allow playback tests, to provide a simple way to verify integration with software.

Home directory

Users of prometheus have a home directory into which they can upload files and modify the contents.  These home directories are populated with files that have a specific meaning, in order to allow quick testing of various features.

The file 'public-file' is publicly readable: all users should be able to see its contents.  The file 'private-file' is only readable by the owner of this home directory.  The directory 'Private' is pre-configured so that only the home directory owner can upload into this directory, or view any content.  The directory `Tape` is configured to simulate tape access: dCache will go through the motions of writing files to tape, but no actual tape drives are used.

Home directories are modifiable by the users – you can upload your own data here.  Please be aware that all modifications (including any upload data) is lost when the service is rebuilt every day.

How to get access?

Just contact me on paul.millar@desy.de.