Blog

Blog

Pulse capacitor constants

you can find the pulse capacitor constants at /gpfs/exfel/exp/SPB/201701/p002012/scratch/user/kuhnm/pcdrs/r0488-r0489-r0490-r0491-r0492-r0493-r0494-r0495
There you can find both formats again: the agipd and the xfel format

Structure:
/channel00/collection/creation_date
                      run_number
           gainlevel_mean
           offset
           slope
           threshold
/channel01/...

where:
- the numbering of the channels is corresponding to the numbering of the raw files, e.g. RAW-...-AGIPD00-S....h5 -> channel 00
- gainlevel_mean: the mean of the digital (gain stage information) value in each gain stage
- offset and slope: results of the fitting for each gain stage area
- threshold: mean of the gainlevel_mean values of two gain stages; used to determine the current gain stage
- collection: some metadata
              - creation_date: when the constants where created
              - run_number: from which run numbers this was created

Raw analog data

Dark offsets, fix gain

calculation: (<raw analog data> - <dark offset>) * <fix gain>

where <fix gain> are the ratios between the gains and were chosen as ratio \frac{H}{M} = 34.5 and ratio \frac{M}{L} = 4.1 

-> this was the correction we gave you during the beamtime

Dark offsets vs pulse capacitor offsets

taking one row of one module of the raw data and compare the smoothness of the dark correction versus the pulse capacitor correction for the medium gain stage

Dark offsets and pulse capacitor gain

calculation: (<raw analog data> - <dark offset>) * <pulse capacitor gain>

Mixed offsets and pulse capacitor gain

Using dark offsets for high gain stage and medium offsets for medium gain stage

calculation (<raw analog data> - <dark offset for high gain stage and pulse capacitor gain for medium gain stage>) * <pulse capacitor gain>

Location

Can be found in the git repo https://stash.desy.de/projects/AGIPD/repos/calibration/browse in branch format_join

There a script with the following name exists: job_scripts/run_analysis.sh

Usage:

Modify the parameter

git_dir: <the direcotry in which git was cloned, e.g. cd /home/kuhnm; git clone <url> -> git_dir=/home/kuhnm/calibration
raw_dir: beamtime of the experiment (the software automatically searches for the correct subdirs)
output_dir: where the results should be written to (the software generates subdirectories)

To generate pulse capacitor constants:
meas_type: pcdrs
run_list: 8 runs (one for each injected row)

To generate dark constants:
meas_type: dark
run_list: 3 runs (one for high medium and low gain)

data_id: is is needed to join the different channels into one file. To distinguish between the constants currently an identifier is added (simply the date on which they where generated)

Run

cd <git_dir>
sh job_scripts/run_analysis.sh

Lysozime in liquid jet

Congratulation to everybody!

AGIPD detected the first proteins delivered by a liquid jet at 1.13MHz repetition rate at XFEL.


Shift plan for User runs 2 and 3

Run 2 21.9.-25.9. (Day shift)

UT ed. 19.09.2017: Updated Ulrich's availability


Name
21.9.22.923.924.9.25.9.


ThuFriSatSunMon
Aschkan
8am-8pm8am-1pmvacationvacationvacation
Ulrich
office

1pm-4pm,

n.A after 4pm

n.A.

(not in HH)

n.A.

(not in HH)

probably n.A.

(not in HH)

Alexander
officeon call8am-8pm(??)on call12am-8pm
Jola

8am-

8am-off8am-8am-
Stephan
tbdtbdtbdtbdtbd
Manuela
vacationvacationvacationvacationvacation
Torsten
officeoffoffoffoffice
Davide
n.A.n.A.n.A.n.A.n.A.
Dominic
n.A.n.A.n.A.n.A.n.A.
Jenny










Run 3 27.09-02.10. (Night shift)

Please add Jola's and Stephan's times as soon as available.


Name
28.9.29.930.901.10.02.10.


ThuFriSatSunMon
Aschkan
vacationvacationvacationvacationvacation
Ulrich
7pm-2amon call7pm-2amon call7pm-2am
Alexander
officenot in HHnot in HHnot in HHvacation
Jola





Stephan





Manuela
vacationvacationvacationvacationvacation
Torsten





Davide
n.A.n.A.n.A.n.A.n.A.
Dominic
n.A.n.A.n.A.n.A.n.A.
Jenny





Droop of digital memory cell part 2

Plotting data for Module 13 (Q4M2) which is the coolest module (~1°C).


Droop of digital memory cell


Data plotted for Pixel(33,133) (Example)

high_gain=h5py.File('/gpfs/exfel/exp/SPB/201730/p900009/scratch/kuhnm/r0428/R0428-AGIPD05-gathered.h5','r')

medium_gain=h5py.File('/gpfs/exfel/exp/SPB/201730/p900009/scratch/kuhnm/r0429/R0429-AGIPD05-gathered.h5','r')

low_gain=h5py.File('/gpfs/exfel/exp/SPB/201730/p900009/scratch/kuhnm/r0430/R0430-AGIPD05-gathered.h5','r')


Temperature of module 5 (Q2M2) is not known but temperatures of other three modules on this quadrant range between 8-10°C

Gathered data bases on pedestal data for three gain stages taken  from run numbers r0428, r0429, r0430.






Medium gain bit


Low gain bit

HG, MG, LG Pedestals

As mentioned yesterday, because of a conflict between XFEL DAQ and the timing of the current source scans, it is not possible to take dynamic range scans here at XFEL for the time being. As a consequence, this means that we cannot measure or calculate the pedestals of medium and low gain.

Therefore, we decided to test, if it is possible to switch gain during the integration time in order to get the pedestal for medium and low gain.

For MG:

  • EGSGAT has to to be changed from 14 to 2
  • G1SGAT has to be changed from 16 to 4

For LG:

  • Changes for MG plus
  • G2SGAT has to be changed from 15 to 3


Please note that the point we obtain here is the interception point on the y-axis of the dynamic range scan.

Since we force gain switching here without charge an offset between the extrapolated interseption point and the point obtained from this measurement is expected but has to be investigated in more detail.



Below you see histograms of analog and digital data for module index 5 (Q2M2). (Module index given in the title of the plots is wrong)

Data location

Test data for implementation into calibration infrastructure can be found here:

/gpfs/exfel/exp/SPB/201730/p900009/raw/

HG: r0428

MG: r0429

LG: r0430

Status

Problem with one of the XFEL modules. No beam available. Not sure yet if there will be any by the beginning of the shift.

Currently testing patterns for acquisition of medium- and low gain pedestals with Jola and Davide.


General status

Water rings from capillary monitored.

Water jet not found yet (6am). Once it was found, the next step would be to inject samples...

Change of calibration strategy

Email to: "Barty Anton" <anton.barty@desy.de>; "Adrian Mancuso" <adrian.mancuso@xfel.eu>; "Steffen Hauf" <steffen.hauf@xfel.eu>; "Heinz Graafsma" <heinz.graafsma@desy.de>; "Chapman Henry" <henry.chapman@desy.de>; "Sasa Bajt" <sasa.bajt@cfel.de>; "steve aplin" <steve.aplin@cfel.de>; "Mariani Valerio" <valerio.mariani@desy.de>; "Jolanta Sztuk-Dambietz" <jolanta.sztuk@xfel.eu>; "Markus Kuster" <markus.kuster@xfel.eu>; "Greiffenberg Dominic" <dominic.greiffenberg@psi.ch>; "davide mezza" <davide.mezza@psi.ch>;


Dear all,

after today's analysis&calibration meeting in the "green area", we internally sat together and generated following plan which I would like to share with you.

As most of you know, the timing scheme of the current source dynamic range scan does not cope very well with the XFEL timing. Therefore, it is currently not possible to acquire current source scans at the beam line. (To give you a better understanding of how such a scan should look like, I attached a scan of an arbitrary pixel to this email.) In order to correct the raw images in medium and low gain, the dynamic range scan is essential. Therefore, we decided to fall back to the dynamic range scans we previously obtained at FS-DS labs.

In principle, from the dynamic range scan one obtains both, gains and offsets of medium and low gain. Unlike the gain, the offsets depend very strongly on temperature (and other things), so, while it makes at least some sense to use the gains we obtained in our lab, it makes basically no sense to use the offsets. (Please note that the current source scans we took in our lab failed for some pixels and sometimes even whole asics, even though the pixels and ASICs are actually good. To explain why would exceed the scope of this email but we are happy to explain it in detail if you like.)
We decided to split the calibration into following tasks:



Gain calibration
Manuela and Torsten are working hard to combine the current source data from our lab with x-ray gain data taken before in our lab. This step gives us the possibility to have calibration parameters in place asap. Once this is working and the calibration parameters can be applied to raw data properly, in a second step they will feed in the Cu x-ray data we obtained at SPB in 30 bunch mode.

Offset calibration in three gain stages
In order to provide reliable offset data for medium and low gain, we plan following approach:
We (Davide) prepared two additional patterns which in principle take dark data as usual. The difference is that they switch gain during the integration period and by that we get the offset of the medium and low gain, respectively. After chats with Adrian and Jola we agreed that the best time to run these scans would be 1-2 hours before the start of the shift, especially as Software- and ITDM people should be around in the event of technical problems. Therefore, I suggest to meet at SPB around 6pm unless you have any objections.

We are convinced that this is currently the most reasonable way to determine the offsets of medium and low gain for the time being (and maybe even in future (tbc)). (Remember that the offset of the high gain is obtained from conventional dark data.)


We  will provide you a block diagram of the new calibration approach tomorrow evening.

Best,


Aschkan (on behalf of the AGIPD calibration team)

Optimizing delays

The blue line shows the mean of a ROI of module 4/Memory cell 4. The red plot is shows the  mean of the same ROI but for the next memory cell 5 for which we do not expect to see photons. On the x-axis the train number is plotted, on the y-axis the ADU value is plotted. While the acquisition was running, the delay of the clock and control trigger was moved to scan the position of the incoming pulse inside the integration window in order to place it at the beginning of the window. The result is that the initial position at 1941789clks (*9.23ns) is already well in the beginning of the window. The delay is changed from 1941777 (train 0) to 1941794  (train 1500) in steps of 1clk/10sec. (100 trains).

Data is available here: /gpfs/p002012/raw/r0002/gpfs/p002012/raw/r0007


Shift plan first user operation
Name
14.9.15.916.917.9.18.9.19.9


ThuFriSatSunMon
Aschkan
4pm-8am5pm-2amon call7pm-6pm-3am
Ulrich
officeon callon call7pm-1am

Alexander
officeon call10pm-4amon call

Jola

8am-11am

5pm-1am

5pm-1am5pm-1amon call5pm-1pm
Stephan
1am-8am1am-8am1am-8am1am-8am1am-8am1am-8am
Manuela
4pm-7pm-tbdtbdtbdtbd
Torsten
4pm-7pm-tbdtbdtbdtbd
Davide
4pm-tbdtbdtbdtbdtbd
Dominic
4pm-tbdtbdtbdtbdtbd
Jenny







Shift Plan for second user operation will be published tonight.


Shift plan in preparation

Dear all,

I wrote a rough shift plan for the next days but I still wait for questions answered by XFEL. For instance, what is planned during day shifts (no beam) and to what extend are detector people needed? This could impact the plan significantly.
However, please note that the beam is available during the nights and I kindly ask you to be available.
Furthermore, it would be good if more people than only Ulrich, Alexander and I can operate the system. Therefore, I will try to schedule a training for tomorrow afternoon but this I also have to discuss with Jola as it also depends on other tasks scheduled for tomorrow.
You will get a detailed plan by tomorrow lunch time.

Best,
Aschkan

Bunch pattern for ~30 pulses created

Jola's Email from tonight:

PS: Igor was at XFLE today afternoon. We manage to create bunch structure pattern with vetos which maybe can allow us 

to get 28 or 29 x-ray pulses running detector with 2xrays_60cbi.txt pattern. We could not test it with x-rays, beacuase there was no xrays,

but thats all what we have at the moment.“

XFEL documentation

All documentations for XFEL is available here:
https://in.xfel.eu/readthedocs/

Interesting for us is the data analysis part:
https://in.xfel.eu/readthedocs/docs/data-analysis-user-documentation/en/latest/