Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


THIS ONLY WORKS ON DATA TAKEN AT XFEL BEFORE MARCH 2018

Run Checks

Inside the calibration framework a sanity check script for data coming from XFEL is available:

Code Block
languagebash
python src/sanity_checks.py --instrument_cycle <instrument_cycle> --beamtime <beamtime> --run <run_number>

for example like this:

Code Block
languagebash
python src/sanity_checks.py --instrument_cycle 201730 --beamtime p900009 --run 709


Currently implemented test

Code Block
languagebash
% python src/sanity_checks.py -h                                                    
usage: sanity_checks.py [-h] --instrument_cycle INSTRUMENT_CYCLE --beamtime
                        BEAMTIME --run RUN

optional arguments:
  -h, --help            show this help message and exit
  --instrument_cycle INSTRUMENT_CYCLE
                        The instrument_cycle the beamtime was taken, e.g
                        201701
  --beamtime BEAMTIME   The beamtime to check
  --run RUN             The run to check

Test descriptions:
--------------------------------------------------------------------------------
| test_data_tzeros:    Check if additional data entries are trailing zeros     |
|------------------------------------------------------------------------------|
| test_train_id_shift: Checks if the first train id value is equal for all     |
|                      modules                                                 |
|------------------------------------------------------------------------------|
| test_n_seqs_equal:   Tests that all modules have the same number of          |
|                      sequences                                               |
|------------------------------------------------------------------------------|
| test_train_id_diff:  Checks if the trainId is monotonically increasing       |
|------------------------------------------------------------------------------|
| test_dims_image:     Checks if the first dimension is equal for all datasets |
|                      contained in 'image' (per module and per seq)           |
|------------------------------------------------------------------------------|
| test_dims_trailer:   Checks if the first dimension is equal for all          |
|                      datasets contained in 'trailer' (per module and per seq)|
|------------------------------------------------------------------------------|
| test_train_loss:     Checks for missing entries in trainId                   |
|------------------------------------------------------------------------------|
| test_train_id_tzero: Checks number of placeholder in trainId and if they are |
|                      always at the end                                       |
|------------------------------------------------------------------------------|
| test_data_vs_pulsec: Checks if the sum of the pulseCount entries is          |
|                      corresponding to the data                               |
|------------------------------------------------------------------------------|
| test_n_train_equal:  Checks if number of trains is equal for all module      |
|                      (per seq)                                               |
|------------------------------------------------------------------------------|
| test_train_id_zeros: Checks if trainId containes zeros which are not at the  |
|                      end                                                     |
|------------------------------------------------------------------------------|
| test_dims_header:    Checks if the first dimension is equal for all          |
|                      datasets contained in 'header' (per module and per seq) |
|------------------------------------------------------------------------------|
| test_train_id_equal: Checks if the train ids taken from detector, header     |
|                      and trailer are equal (per module)                      |
|------------------------------------------------------------------------------|
| test_data_vs_tr_id:  Checks if the dimension of the image trainId is         |
|                      corresponding to the data)                              |
--------------------------------------------------------------------------------


Examples

Run without failed checks:

Code Block
% python src/sanity_checks.py --instrument_cycle 201701 --beamtime p002066 --run 110
test_n_seqs_equal (__main__.SanityChecks) ... ok
test_data_tzeros (__main__.SanityChecks) ... ok
test_data_vs_pulsec (__main__.SanityChecks) ... ok
test_data_vs_tr_id (__main__.SanityChecks) ... ok
test_dims_header (__main__.SanityChecks) ... ok
test_dims_image (__main__.SanityChecks) ... ok
test_dims_trailer (__main__.SanityChecks) ... ok
test_n_train_equal (__main__.SanityChecks) ... ok
test_train_id_diff (__main__.SanityChecks) ... ok
test_train_id_equal (__main__.SanityChecks) ... ok
test_train_id_shift (__main__.SanityChecks) ... ok
test_train_id_tzero (__main__.SanityChecks) ... ok
test_train_id_zeros (__main__.SanityChecks) ... ok
test_train_loss (__main__.SanityChecks) ... ok

----------------------------------------------------------------------
Ran 14 tests in 3.188s

OK


Run with failed checks:

Code Block
languagebash
% python src/sanity_checks.py --instrument_cycle 201730 --beamtime p900009 --run 709
test_n_seqs_equal (__main__.SanityChecks) ... ok
test_data_tzeros (__main__.SanityChecks) ... ok
test_data_vs_pulsec (__main__.SanityChecks) ... FAIL
test_data_vs_tr_id (__main__.SanityChecks) ... ok
test_dims_header (__main__.SanityChecks) ... ok
test_dims_image (__main__.SanityChecks) ... ok
test_dims_trailer (__main__.SanityChecks) ... ok
test_n_train_equal (__main__.SanityChecks) ... FAIL
test_train_id_diff (__main__.SanityChecks) ... ok
test_train_id_equal (__main__.SanityChecks) ... ok
test_train_id_shift (__main__.SanityChecks) ... FAIL
test_train_id_tzero (__main__.SanityChecks) ... FAIL
test_train_id_zeros (__main__.SanityChecks) ... ok
test_train_loss (__main__.SanityChecks) ... FAIL

======================================================================
FAIL: test_data_vs_pulsec (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "src/sanity_checks.py", line 482, in test_data_vs_pulsec
    self.fail(msg)
AssertionError: 
PulseCount and data shape do not match for the following
channels and sequences (pulseCount sum vs data shape):
Channel 00, sequence 0 (18500 vs 18512)
Channel 00, sequence 1 (18500 vs 18512)
Channel 00, sequence 2 (8362 vs 8368)
Channel 01, sequence 0 (18500 vs 18512)
Channel 01, sequence 1 (18500 vs 18512)
Channel 02, sequence 0 (18500 vs 18512)
Channel 02, sequence 1 (18500 vs 18512)
Channel 02, sequence 2 (8362 vs 8368)
Channel 03, sequence 0 (18500 vs 18512)
Channel 03, sequence 1 (18500 vs 18512)
Channel 03, sequence 2 (8362 vs 8368)
Channel 04, sequence 0 (18500 vs 18512)
Channel 04, sequence 1 (18500 vs 18512)
Channel 04, sequence 2 (8362 vs 8368)
Channel 05, sequence 0 (18500 vs 18512)
Channel 05, sequence 1 (18500 vs 18512)
Channel 05, sequence 2 (8362 vs 8368)
Channel 06, sequence 0 (18500 vs 18512)
Channel 06, sequence 1 (18500 vs 18512)
Channel 06, sequence 2 (8362 vs 8368)
Channel 07, sequence 0 (18500 vs 18512)
Channel 07, sequence 1 (18500 vs 18512)
Channel 07, sequence 2 (8362 vs 8368)
Channel 08, sequence 0 (18500 vs 18512)
Channel 08, sequence 1 (18500 vs 18512)
Channel 08, sequence 2 (8362 vs 8368)
Channel 09, sequence 0 (18500 vs 18512)
Channel 09, sequence 1 (18500 vs 18512)
Channel 09, sequence 2 (8362 vs 8368)
Channel 10, sequence 0 (18500 vs 18512)
Channel 10, sequence 1 (18500 vs 18512)
Channel 10, sequence 2 (8362 vs 8368)
Channel 11, sequence 0 (18500 vs 18512)
Channel 11, sequence 1 (18500 vs 18512)
Channel 11, sequence 2 (8362 vs 8368)
Channel 12, sequence 0 (18500 vs 18512)
Channel 12, sequence 1 (18500 vs 18512)
Channel 12, sequence 2 (8362 vs 8368)
Channel 13, sequence 0 (18500 vs 18512)
Channel 13, sequence 1 (18500 vs 18512)
Channel 13, sequence 2 (8362 vs 8368)
Channel 14, sequence 0 (18500 vs 18512)
Channel 14, sequence 1 (18500 vs 18512)
Channel 14, sequence 2 (8362 vs 8368)
Channel 15, sequence 0 (18500 vs 18512)
Channel 15, sequence 1 (18500 vs 18512)
Channel 15, sequence 2 (8362 vs 8368)

======================================================================
FAIL: test_n_train_equal (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "src/sanity_checks.py", line 290, in test_n_train_equal
    self.fail(msg)
AssertionError: 
Not all modules have the same number of trains for seq 2
(n_trains=[128 112 128 128 128 128 128 128 128 128 128 128 128 128 128 128])

======================================================================
FAIL: test_train_id_shift (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "src/sanity_checks.py", line 421, in test_train_id_shift
    self.assertEqual(len(unique), 1, msg)
AssertionError: 
Channels with shifted first train id: [0 1 2 3 4 5 6 7]
channel 00: 1532338073
channel 01: 1532338073
channel 02: 1532338073
channel 03: 1532338073
channel 04: 1532338073
channel 05: 1532338073
channel 06: 1532338073
channel 07: 1532338073
channel 08: 1532338072
channel 09: 1532338072
channel 10: 1532338072
channel 11: 1532338072
channel 12: 1532338072
channel 13: 1532338072
channel 14: 1532338072
channel 15: 1532338072


======================================================================
FAIL: test_train_id_tzero (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "src/sanity_checks.py", line 540, in test_train_id_tzero
    self.fail(msg)
AssertionError: 
Sequence 2 has not the same number of trailing zeros for all channels
([15, 0, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15])

======================================================================
FAIL: test_train_loss (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "src/sanity_checks.py", line 626, in test_train_loss
    self.fail(msg)
AssertionError: 
Channel 01, seq 2: Train loss found at indices:
	idx 31: ... 1532338601 1532338602 1532338604 1532338605 ...

----------------------------------------------------------------------
Ran 14 tests in 25.875s

FAILED (failures=5)

 

Add new tests

  • in class Sanity Checks: add a new method called "test_<your test name>"
  • add the name and the description of the new method into the description dictionary (global variable)
    • only tests contained in this description dictionary are actually run