| Table of Contents |
|---|
THIS ONLY WORKS ON DATA TAKEN AT XFEL BEFORE MARCH 2018
Run Checks
Inside the calibration framework a sanity check script for data coming from XFEL is available:
| Code Block | ||
|---|---|---|
| ||
python src/sanity_checks.py --instrument_cycle <instrument_cycle> --beamtime <beamtime> --run <run_number> |
for example like this:
| Code Block | ||
|---|---|---|
| ||
python src/sanity_checks.py --instrument_cycle 201730 --beamtime p900009 --run 709 |
Currently implemented test
| Code Block | ||
|---|---|---|
| ||
% python src/sanity_checks.py -h
usage: sanity_checks.py [-h] --instrument_cycle INSTRUMENT_CYCLE --beamtime
BEAMTIME --run RUN
optional arguments:
-h, --help show this help message and exit
--instrument_cycle INSTRUMENT_CYCLE
The instrument_cycle the beamtime was taken, e.g
201701
--beamtime BEAMTIME The beamtime to check
--run RUN The run to check
Test descriptions:
--------------------------------------------------------------------------------
| test_data_tzeros: Check if additional data entries are trailing zeros |
|------------------------------------------------------------------------------|
| test_train_id_shift: Checks if the first train id value is equal for all |
| modules |
|------------------------------------------------------------------------------|
| test_n_seqs_equal: Tests that all modules have the same number of |
| sequences |
|------------------------------------------------------------------------------|
| test_train_id_diff: Checks if the trainId is monotonically increasing |
|------------------------------------------------------------------------------|
| test_dims_image: Checks if the first dimension is equal for all datasets |
| contained in 'image' (per module and per seq) |
|------------------------------------------------------------------------------|
| test_dims_trailer: Checks if the first dimension is equal for all |
| datasets contained in 'trailer' (per module and per seq)|
|------------------------------------------------------------------------------|
| test_train_loss: Checks for missing entries in trainId |
|------------------------------------------------------------------------------|
| test_train_id_tzero: Checks number of placeholder in trainId and if they are |
| always at the end |
|------------------------------------------------------------------------------|
| test_data_vs_pulsec: Checks if the sum of the pulseCount entries is |
| corresponding to the data |
|------------------------------------------------------------------------------|
| test_n_train_equal: Checks if number of trains is equal for all module |
| (per seq) |
|------------------------------------------------------------------------------|
| test_train_id_zeros: Checks if trainId containes zeros which are not at the |
| end |
|------------------------------------------------------------------------------|
| test_dims_header: Checks if the first dimension is equal for all |
| datasets contained in 'header' (per module and per seq) |
|------------------------------------------------------------------------------|
| test_train_id_equal: Checks if the train ids taken from detector, header |
| and trailer are equal (per module) |
|------------------------------------------------------------------------------|
| test_data_vs_tr_id: Checks if the dimension of the image trainId is |
| corresponding to the data) |
-------------------------------------------------------------------------------- |
Examples
Run without failed checks:
| Code Block |
|---|
% python src/sanity_checks.py --instrument_cycle 201701 --beamtime p002066 --run 110 test_n_seqs_equal (__main__.SanityChecks) ... ok test_data_tzeros (__main__.SanityChecks) ... ok test_data_vs_pulsec (__main__.SanityChecks) ... ok test_data_vs_tr_id (__main__.SanityChecks) ... ok test_dims_header (__main__.SanityChecks) ... ok test_dims_image (__main__.SanityChecks) ... ok test_dims_trailer (__main__.SanityChecks) ... ok test_n_train_equal (__main__.SanityChecks) ... ok test_train_id_diff (__main__.SanityChecks) ... ok test_train_id_equal (__main__.SanityChecks) ... ok test_train_id_shift (__main__.SanityChecks) ... ok test_train_id_tzero (__main__.SanityChecks) ... ok test_train_id_zeros (__main__.SanityChecks) ... ok test_train_loss (__main__.SanityChecks) ... ok ---------------------------------------------------------------------- Ran 14 tests in 3.188s OK |
Run with failed checks:
| Code Block | ||
|---|---|---|
| ||
% python src/sanity_checks.py --instrument_cycle 201730 --beamtime p900009 --run 709
test_n_seqs_equal (__main__.SanityChecks) ... ok
test_data_tzeros (__main__.SanityChecks) ... ok
test_data_vs_pulsec (__main__.SanityChecks) ... FAIL
test_data_vs_tr_id (__main__.SanityChecks) ... ok
test_dims_header (__main__.SanityChecks) ... ok
test_dims_image (__main__.SanityChecks) ... ok
test_dims_trailer (__main__.SanityChecks) ... ok
test_n_train_equal (__main__.SanityChecks) ... FAIL
test_train_id_diff (__main__.SanityChecks) ... ok
test_train_id_equal (__main__.SanityChecks) ... ok
test_train_id_shift (__main__.SanityChecks) ... FAIL
test_train_id_tzero (__main__.SanityChecks) ... FAIL
test_train_id_zeros (__main__.SanityChecks) ... ok
test_train_loss (__main__.SanityChecks) ... FAIL
======================================================================
FAIL: test_data_vs_pulsec (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
File "src/sanity_checks.py", line 482, in test_data_vs_pulsec
self.fail(msg)
AssertionError:
PulseCount and data shape do not match for the following
channels and sequences (pulseCount sum vs data shape):
Channel 00, sequence 0 (18500 vs 18512)
Channel 00, sequence 1 (18500 vs 18512)
Channel 00, sequence 2 (8362 vs 8368)
Channel 01, sequence 0 (18500 vs 18512)
Channel 01, sequence 1 (18500 vs 18512)
Channel 02, sequence 0 (18500 vs 18512)
Channel 02, sequence 1 (18500 vs 18512)
Channel 02, sequence 2 (8362 vs 8368)
Channel 03, sequence 0 (18500 vs 18512)
Channel 03, sequence 1 (18500 vs 18512)
Channel 03, sequence 2 (8362 vs 8368)
Channel 04, sequence 0 (18500 vs 18512)
Channel 04, sequence 1 (18500 vs 18512)
Channel 04, sequence 2 (8362 vs 8368)
Channel 05, sequence 0 (18500 vs 18512)
Channel 05, sequence 1 (18500 vs 18512)
Channel 05, sequence 2 (8362 vs 8368)
Channel 06, sequence 0 (18500 vs 18512)
Channel 06, sequence 1 (18500 vs 18512)
Channel 06, sequence 2 (8362 vs 8368)
Channel 07, sequence 0 (18500 vs 18512)
Channel 07, sequence 1 (18500 vs 18512)
Channel 07, sequence 2 (8362 vs 8368)
Channel 08, sequence 0 (18500 vs 18512)
Channel 08, sequence 1 (18500 vs 18512)
Channel 08, sequence 2 (8362 vs 8368)
Channel 09, sequence 0 (18500 vs 18512)
Channel 09, sequence 1 (18500 vs 18512)
Channel 09, sequence 2 (8362 vs 8368)
Channel 10, sequence 0 (18500 vs 18512)
Channel 10, sequence 1 (18500 vs 18512)
Channel 10, sequence 2 (8362 vs 8368)
Channel 11, sequence 0 (18500 vs 18512)
Channel 11, sequence 1 (18500 vs 18512)
Channel 11, sequence 2 (8362 vs 8368)
Channel 12, sequence 0 (18500 vs 18512)
Channel 12, sequence 1 (18500 vs 18512)
Channel 12, sequence 2 (8362 vs 8368)
Channel 13, sequence 0 (18500 vs 18512)
Channel 13, sequence 1 (18500 vs 18512)
Channel 13, sequence 2 (8362 vs 8368)
Channel 14, sequence 0 (18500 vs 18512)
Channel 14, sequence 1 (18500 vs 18512)
Channel 14, sequence 2 (8362 vs 8368)
Channel 15, sequence 0 (18500 vs 18512)
Channel 15, sequence 1 (18500 vs 18512)
Channel 15, sequence 2 (8362 vs 8368)
======================================================================
FAIL: test_n_train_equal (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
File "src/sanity_checks.py", line 290, in test_n_train_equal
self.fail(msg)
AssertionError:
Not all modules have the same number of trains for seq 2
(n_trains=[128 112 128 128 128 128 128 128 128 128 128 128 128 128 128 128])
======================================================================
FAIL: test_train_id_shift (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
File "src/sanity_checks.py", line 421, in test_train_id_shift
self.assertEqual(len(unique), 1, msg)
AssertionError:
Channels with shifted first train id: [0 1 2 3 4 5 6 7]
channel 00: 1532338073
channel 01: 1532338073
channel 02: 1532338073
channel 03: 1532338073
channel 04: 1532338073
channel 05: 1532338073
channel 06: 1532338073
channel 07: 1532338073
channel 08: 1532338072
channel 09: 1532338072
channel 10: 1532338072
channel 11: 1532338072
channel 12: 1532338072
channel 13: 1532338072
channel 14: 1532338072
channel 15: 1532338072
======================================================================
FAIL: test_train_id_tzero (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
File "src/sanity_checks.py", line 540, in test_train_id_tzero
self.fail(msg)
AssertionError:
Sequence 2 has not the same number of trailing zeros for all channels
([15, 0, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15])
======================================================================
FAIL: test_train_loss (__main__.SanityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
File "src/sanity_checks.py", line 626, in test_train_loss
self.fail(msg)
AssertionError:
Channel 01, seq 2: Train loss found at indices:
idx 31: ... 1532338601 1532338602 1532338604 1532338605 ...
----------------------------------------------------------------------
Ran 14 tests in 25.875s
FAILED (failures=5) |
Add new tests
- in class Sanity Checks: add a new method called "test_<your test name>"
- add the name and the description of the new method into the description dictionary (global variable)
- only tests contained in this description dictionary are actually run