This page collects and comments pragmatic example Matlab scripts to access data stored on the FLASH DAQ. The main documentation of how to access the DAQ is maintained by MCS 4. Alternatives to access DAQ data are summarised in our how-to.
The scripts use the Mex functions
daq_read_svr provided by DESY's group MCS 4 to access the DAQ. These functions are provided for a specific version of MAC OS and GNU/Linux, here Ubuntu 16.04 64 bit, only. Also, support is limited to Matlab release R2016b.
For full functionality these Mex functions also need access to the MCS 4 file system exported via NFS. Access to that file system is limited to nodes run by MCS 4. Therefore, to use the full functionality you are limited to run your code on
2 or the workstations
Limited functionality of the Mex functions is offered on other GNU/Linux, Ubuntu 16.04, machines on the DESY network, provided the package
doocs-matlab-extern-func is installed. In the Flash hall these are
In any case, make sure,
/local/lib is on your Matlab path.
Both Mex functions,
daq_read_svr, take as a parameter either a filename or a cell array of strings. The choice of parameter type determines, which range of functionality is provided. Neither is a subset of the other. Even though the cell array of strings seems a very natural start, be prepared to have to switch to XML-ish file definitions to get at the desired DAQ data (the XML-ish definitions of the requested data also can be created, or used by the DAQdataGUI).
daq_read requires the DAQ data files in
.raw format to be available on the file system of the computer, where you run the script.
daq_read_svr uses the DAQ data servers running on
flashdaqsvr[2-6]. You preferably choose a server running locally, by setting the environment variable
TTF2_DAQ_SERVERS accordingly. So, on
flashlxuser1 you would set for example in your Matlab code:
setenv('TTF2_DAQ_SERVERS', ['TTF2.DAQ/DAQ.SERVER4/DAQ.DATA.SVR/:' ... 'TTF2.DAQ/DAQ.SERVER3/DAQ.DATA.SVR/:TTF2.DAQ/DAQ.SERVER2/DAQ.DATA.SVR/:' ... 'TTF2.DAQ/DAQ.SERVER1/DAQ.DATA.SVR/]');
Both Mex functions suffer from a memory leak. Therefore try and limit the number of calls to these functions in your code. Also, for efficiency it is advised to request a few longer intervals of data than many shorter chunks.
Often the specification of the run number is mandatory. To figure out what run number your data belong to, use the DAQdataGUI. Sometimes however, the specification of the run number causes the Mex function call to fail. You have to find out what works by trial and error.
Less frequently you need to specify the standard data location, ie. the path to the
.raw files on the DAQ server, called the data directory. The files' default location depends on the DAQ stream, alias experiment name, your data have been recorded with (see the table below). Sometimes you also need to specify the location of the run catalogue, which depends on the DAQ used to record the data, called the run catalogue directory (see table).
|Experiment Name||Default Data Directory||Catalogue Directory|
Single Channel Access
Rational: During a beamtime you might want to confirm that data are recorded by the FLASH DAQ. Or you might want to access pulse ID synchronised data, which are not accessible through the GMD provider server. In a situation like this, simple access to just one DAQ channel helps. This paragraph shows, how to retrieve a single channel of DAQ data for a given time period using the Matlab
The term channel is DAQ lingo and corresponds (almost) to what is called a property elsewhere in DOOCS. A mapping of channels to DOOCS properties and an explanation of their meaning of the PBD DAQ (Photon Beam Diagnostics DAQ) is given here. Which channels are recorded with the EXP2 DAQ depends on its configuration.
DaqRead the requirements listed above must be met and the M files
DaqServerReader.m must be on the Matlab search path. We suggest, you download the files from their repository to the place where you keep your Matlab libraries, ie. the Matlab userpath (type
userpath into the Matlab command window to find out, where that is).
To check your set-up enter at the Matlab command window:
>> which DaqRead >> which daq_read
To get help with the use of
>> help DaqRead
or more specifically for help on a method,
data in this example, enter:
>> help DaqRead.data
To discover if the DAQ has recorded ADC data you could plot a single sampling point versus time or more precisely versus the pulse ID:
>> [trace, ids]= DaqRead().fl1user2().data('2017-07-25 17:40', 1, 'FLASH.FEL/ADC.SIS.BL2/EXP2.CH00'); >> plot(ids - ids(1), trace(2000,:), 'b.');
DaqRead object is configured to use the FLASH PBD DAQ by default. Therefore you have to specify the DAQ, the FLASH1 USER2 DAQ in this case. The returned matrix
trace contains the recorded ADC data, the sampling points in the columns and a row for each pulse.
The next example shows how to use the default of
DaqRead for the PBD DAQ. To see the SASE electron energy over 10 minutes recorded with the PBD DAQ you would enter for example:
>> sase= DaqRead().data('2016-03-28 10:59', 10, 'PBD.PHFLUX/BDA.ENERGYPULSE.FF'); >> imagesc(sase);
Here, the energy pulse
sase is sliced to account for a repetition rate of 200 kHz; only at every 5th column a bunch is expected.
To investigate which data are recorded with a specific DAQ channel at a certain point in time try (intentionally without a semicolon at the end):
>> structs= DaqRead().structs('2016-03-28 10:59', 0.1, 'PBD.PHOTONWL.ML/WAVE_LENGTH')
This request gives you just six seconds (0.1 minutes) of data as an array of structures. To then display just the spectrum use:
The examples given above will cease to function when the
.raw files are deleted from disk.
DaqRead.structs methods as a side effect write the XML-ish data definition to the
/tmp directory with the filename concatenated from user ID and process ID. The file is not deleted automatically, to allow for the analysis of failing calls to the Mex functions
If you want to reuse a
DaqRead object or you don't like the cascading of messages, the previous examples could be rewritten using single method invocations per statement as:
>> reader= DaqRead; % create a reader object >> reader.local; % configure it to use local access, ie. daq_read >> reader.pbd; % configure it explicitly to use the default PBD DAQ >> wl= reader.structs('2016-03-28 10:59', 1, 'PBD.PHOTONWL.ML/WAVE_LENGTH'); >> sase= reader.data('2016-03-28 10:59', 1, 'PBD.PHFLUX/BDA.ENERGYPULSE.FF'); >> disp(wl); >> % ...
The calls of
DaqRead.structs fail fairly frequently due to failures of the calls to the underlying Mex functions
daq_read_svr respectively. These functions return very limited error messages. Most of the error output goes to the operating system console, which can not be captured by Matlab. Therefore, if your calls error, consult the output on the operating system console on which you have started Matlab. If you start Matlab using a GUI, you don't get to see the error output at all.
For online data analysis you might not want to access the DAQ after they have been written to Disk as you do with the approach described on this page. A more appropriate choice would be to use the GMD provider server.
- Source code repository of the Matlab class
DaqReadto read single channels of the Flash DAQ
- DaqServerReader.m : Wiki detailing the use of Mex functions
and here the list with server names:
|Experiment Name||Default Data Directory||Catalogue Directory||server|