Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

 

as seen from a machine at the beamline

Code Block
languagebash
titlebeamline file-system
/gpfs/
├── local
├── current
|    ├── raw
|    ├── processed
|    ├── shared
|    └── scratch_bl
└── commissioning
    ├── raw
    ├── processed
    ├── shared
    ├──└── scratch_bl
 
/common/
├──└── p01<beamline>
├── p02.1
├── p02.2
├── p03
├── p04
├── p05
├── p06
├── p07
├── p08
├── p09
├── p10
└── p11
Code Block
languagebash

/bl_documents


as seen from maxwell

Code Block
titlecore file-system
/asap3/
└── petra3<facility>
    └── gpfs
        ├── <beamline>
		|	├── <year>
		|		├── data
	    |       |   ├── └── <beamtime-ID>
    	|       |         ├── raw
        |	    |        ├── processed
        |    	|        ├── shared
        |       |	     └── scratch_cc
		|		└── commissioning
        |        	└── <commissioning-ID>tag>
        |            ├──	|── raw
        |            	├── processed
        |            	├── shared
        |            	└── scratch_cc
        └── common
            ├──└── p01
            ├── p02.1
            ├── p02.2
            ├── p03
            ├── p04
            ├── p05
            ├── p06
            ├── p07
            ├── p08
            ├── p09
            ├── p10
            └── p11
Description:
<beamline>


Description

  • 'local': 
    • A
3TB
    • 3 TiB local share of the beamline file-system which stays there independently of the beamtimes
    • It is managed by the beamlines themselves
    • There is no syncing across to the core file-system.
    • It serves as a local buffer.
    • Anybody on the beamline can read/write from/to it
    • Cannot be accessed from the core file-system
  • 'current':
    • Non-permanent share from the beamline file-system
    • It will appear when a beamtime is started and disappear if it is stopped
    • Meaning of the different directories:

      • raw: 
    for
          • For raw data
          • will
    be
          • be 
            • migrated into the core file-system
    will be
            • written to tape
    will be
            • shown in the Gamma-portal
        • processed
      for
            • For meaningful processed data
            • will
      be
            • be 
              • migrated into the core file-system
      will be
              • written to tape
      will be
              • shown in the Gamma-portal
          • shared
        for
              • For user specific macros, scripts, metadata, text-files,...
              • will be
                • migrated into the core file-system
        will be
                • written to tape
        will be
                • shown in the Gamma-portal
            • scratch
          meant for
                • Meant for 
                  • temporarily data
          or
          for
                  • data
          where in advance
                  • it is not
          knows
                  • known in advance if
          it
                  • is meaningful or not.
           It
                  • That data can be written here and if its meaningful copied into 'processed' afterwards.
                • will not be migrated into the core file-system
          • 'commissioning'
            same
              • Same as 'current' but for
            the
              • commissioning runs
              • Commissioning runs are limited to a 1 TiB hard quota
            • 'common':
            mounted
              • Mounted read-only in the beamline space
            , its
              • Its world-readable meaning that all beamlines can read it
              • Meant for documentation, macros, ... provided to the users by the beamline staff
            if a user
              • If users has to edit something, like a macro,
            he
              • they can copy it into 'shared'
            can
              • Can be changed from the core side by the beamline staff
            • bl_documents:
              • Mounted read-write in the beamline space with 1TiB hard quota per beamline
              • only the beamline specific part is mounted
              • stays there independent of the beam times
              • has 2 snapshots daily, 28 (4 weeks) are kept.
              • For scripts and documentation
              • Is not visible from Maxwell and will remain so


            Description

            • Access only with a valid DESY or Science Account
            • '<facility>'
              • Determines the facility, where the data has been acquired
              • Supported facilities
                • petra3
                • flash
                • spec.instruments
              • The remaining directory structure is identical between both facilities