CALICE
Space shortcuts
Page tree
Jiri Kvasnicka posted on 03. Oct. 2018 10:44h - last edited by Jiri Kvasnicka on 29. May. 2019 18:34h

EUDAQ architecture

EUDAQ_ahcal_diagram

Components:

  • Run control GUI: main interface with buttons to init, configure, start and stop. There is a special runcontrol with automatic restart feature: AhcalRunControl
  • Data collector: takes the data from (potentially more) producers and merge them into events. AHCAL has two custom producers:
    • CaliceAhcalBifBxidDataCollector, which combines everything in same BXID together
    • AhcalHodoscopeDataCollector with more debug outputs for combined running with Tokyo hodoscope (functionality is covered by the bxid collector as well)
  • Producer: an EUDAQ module, which talks to the detector device and sens events to the data collectors. Destination data collectors are fully configurable
  • Log collector: collects error messages from all producers
  • Online Monitor: provides an hitmaps and basic data observables. Natively works with pixel planes
  • Data converter plugins: are loaded dynamically when conversion to another format is requested (for example eudaq raw → slcio, eudaq raw → standard event)

Connections between producers and data collectors are defined in the configuration ini file: a list of the destination data collectors is given as a parameter to the producer. 

EUDAQ_DC="dc1,dc3,bxidColl1"

Startup script

The startup script kills previous instances of EUDAQ, as defined in file KILLRUN.local or KILLRUN.sh. all producers are started in separate xterm, so that the outputs don't mix.

Startup script example 
#TODO put the startup script from EUDAQ PC


#!/bin/bash

if [ -f KILLRUN.local ]
then
    sh KILLRUN.local
else
    sh KILLRUN.sh
fi

export RCPORT=44000
export HOSTIP=127.0.0.1
#################  Run control ###################
#xterm -sb -sl 1000000 -T "Runcontrol" -e 'bin/euRun -n RunControl -a tcp://$RCPORT ; read '&
xterm -sb -sl 1000000 -T "Runcontrol" -e 'bin/euRun -n AhcalRunControl -a tcp://$RCPORT ; read '&
sleep 1 # the runcontrol has to start first, otherwise 
#################  Log collector #################
xterm -sb -sl 1000 -geometry 160x24 -T "Logger" -e 'bin/euLog ; read' &
#sleep 1
#################  Data collector #################
xterm -sb -sl 100000 -T "dc1" -e 'bin/euCliCollector -n DirectSaveDataCollector -t dc1 ; read' &
xterm -sb -sl 100000 -geometry 160x24 -T "BXID collector ahcalbif" -e 'bin/euCliCollector -n CaliceAhcalBifBxidDataCollector -t bxidColl1 ; read' &
#xterm -sb -sl 100000 -geometry 160x30 -T "Hodoscope Data collector" -e 'bin/euCliCollector -n AhcalHodoscopeDataCollector -t dc2 | tee -a logs/dc2.log; read' &
#################  Producer #################
#xterm -sb -sl 1000000 -geometry 160x30 -T "Hodoscope" -e 'bin/euCliProducer -n CaliceEasirocProducer -t hodoscope1 -r tcp://$HOSTIP:$RCPOR|tee -a logs/hodoscope1.log && read || read'&
#xterm -sb -sl 1000000 -geometry 160x30 -T "Hodoscope 2" -e 'bin/euCliProducer -n CaliceEasirocProducer -t hodoscope2 -r tcp://$HOSTIP:$RCPOR|tee -a logs/hodoscope2.log && read || read'&
xterm -sb -sl 100000 -geometry 160x24 -T "AHCAL" -e 'bin/euCliProducer -n AHCALProducer -t AHCAL1 |tee -a data/logs/ahcal.log ; read'&
xterm -sb -sl 100000 -geometry 160x24 -T "BIF" -e 'bin/euCliProducer -n caliceahcalbifProducer -t BIF1 |tee -a data/logs/bif.log ; read'&
#################  Online Monitor #################
xterm -sb -sl 100000  -T "Online monitor" -e 'bin/StdEventMonitor -c conf/onlinemonitor.conf --monitor_name StdEventMonitor --reset -r tcp://$HOSTIP:$RCPORT ; read' &

Configuration of AHCAL producer

Example configuration scipt is here:

AHCAL producer configuration section 
[Producer.AHCAL1]
EUDAQ_DC="dc1,dc3,bxidColl1" # names of data collectors, where will be data sent
#FileLEDsettings ="F:\\LEDEUDAQ\\LED1.ini" #if used, Labview will load LED settings from the ini file on the labview computer
# Sleeping time (in seconds) after clicking stop: needed to read
# all the events stored in the Labview data queue
waitsecondsForQueuedEvents = 3 # wait time in the ScReader::OnStop() method. 
IPAddress = "192.168.1.11" # IP address of the labview PC
Port = 5622 # port number defined in the Labview
Reader = "Scintillator" # needs to be "Scintillator" for AHCAL DAQ
WriteRawOutput = 1 # whether to save the raw data in the producer
RawFileName = "data/AhcalRaw/ahcalRaw_Run%06d" # raw filename pattern
WriteRawFileNameTimestamp = 1 # whether to add a timestamp to the name
ColoredTerminalMessages = 1 # whether to use colored output or not. For Linux: yes, windows: no
LdaTrigidOffset = 1 # to compensate differences between TLU (or other trigger number source) and LDA. Eudaq Event counting starts from this number and will be always subtracted from the eudaq event triggerid.
LdaTrigidStartsFrom = 1 # The first expected raw triggerID from LDA in the data stream in case it is not 0. It suppresses error warning for the first event. When set higher then the first event, triggers below the provided number are ignored!
GenerateTriggerIDFrom = 0 # sets from which triggerID number should be data generated (and filled with dummy triggers if necessary). Only works when insert_dummy_packets is enabled and in selected event building modes
InsertDummyPackets = 0 # 1=inserts dummy packets to keep correct sequence of TrigIDs or ReadOutCycles, even if there is not hit registered; 0=do not insert anything
#When RedirectInputFromFile is set, an existing raw raw data file is used as a fake input from detector
#RedirectInputFromFile="/data/201805/ahcalRaw_Run060939__22p05p2018__16p38p52.raw" # Will redirect the input from the file instead of talking to the labview
#EventBuildingMode selects what event building will be used. Uncomment which mode is used (options: "ROC", "TRIGGERID", "BUILD_BXID_ALL", "BUILD_BXID_VALIDATED")
#EventBuildingMode = "ROC" # A complete Readoutcycle will 
#EventBuildingMode = "TRIGGERID"
#EventBuildingMode = "BUILD_BXID_ALL"
EventBuildingMode = "BUILD_BXID_VALIDATED"
#EventNumberingPreference chooses a main numbering method, when both are available (especially in BUILD_BXID_VALIDATE and TRIGGERID event building mode) 
#EventNumberingPreference = "TRIGGERID"
EventNumberingPreference = "TIMESTAMP"
IgnoreLdaTimestamps = 0 # ignores timestamp from the LDA data stream
KeepBuffered = 50 # how many readout cycle will be held in the memory befor sending out. This filters out packets, that come out of order. Necessary for running with 2 wing-LDAs
DebugKeepBuffered = 0 # Debug option: 1=keep all events in producer buffer and don't send the data to data collector. 0=pass packets normally
AHCALBXID0Offset = 2106 # offset from start acquisition Timestamp to BXID0 (in 25ns steps). Varies with AHCAL powerpulsing setting and DIF firmware
#AHCALBXIDWidth = 
AHCALBXIDWidth = 160 # length of the bxid in 25 ns steps. 4us=160 (default), 200ns=8
MaximumTrigidSkip = 30000 # maximum numbers of triggers, that are allowed to skip. If more, treated as bad data
MaximumROCJump = 100 # maximum jump in Readoutcycle number.
ChipidAddBeforeMasking = -1 # number to be added to the chipID: use -1 to count chipid from 0
ChipidKeepBits = 4 # How many bits from the chipid are kept
ChipidAddAdterMasking = 0 # number to be added to the chipid after bit masking
AppendDifidToChipidBitPosition = 8 # bit position where to add the difid to the chip id. Should be 8
MinimumBxid = 1 # minimal accepted bxid. bxid0 has a TDC bug, therefore it makes sense to start from 1
MinimumEventHits = 1 # event will be produced only when number of hits is greater or equal to given number

Values, that usually do need tweaking:

  • EUDAQ_DC
  • EventBuildingMode
  • EventNumberingPreference
  • AHCALBXID0Offset

  • IPAddress

  • trigger numbering parameters:

    • LdaTrigidOffset 

    • LdaTrigidStartsFrom

    • GenerateTriggerIDFrom

    • InsertDummyPackets

LDA time offset

The "LDA offset" is a number of clock ticks from the moment of reception of the start acquisition fast command to the moment, where starts the BXID 0 in the SPIROC chip. This is important for matching trigger timestamps and BXID number in SPIROC. The offset depends on:

  • HDMI cables
  • trigger cable length + electronics latency
  • Powerpulsing configuration

The offset needs to be calibrated each time there is a possible change of timing (different cables for example). 

Easiest way is to use following script from  this project: https://github.com/jkvas/AHCAL-RAWutils, that searches the number of correlated events with respect to different offset value. 

script parameters 
$AHCAL_RAWUTIL_PATH/ahcalBifCorrelation/ahcalbifcorrelate \
--spiroc_raw_file $AHCAL_RAW_FILE \
--trig_data_from_spiroc_raw \
--require_hitbit \
--shift_scan_max 500000 \
--shift_scan_method 1 \
--bxid_length 160 >LDA_offset.txt

The output file contains the tab-separated values (offset, correlation, normalized correlations) and commented lines, one of them with the optimal offset value (which should be used as AHCALBXID0Offset):

#maximum correlation at: 14106 hits:6116261

It is highly recommended to check the graph of number of time correlations vs the offset. Output can be displayed with gnuplot command:

gnuplot # start the gnuplot
plot 'LDA_offset.txt' u 1:3 w lines #type this in gnuplot to get a plot

Typical results without validation look like this: (full range on the left and zoomed with right mouse button click on the right):

When validation is on, the peak has few kings at the top. Zoomed position of the peak to be used is on the right side (very low statistics in this plot):

If there is not visible peak, check: --shift_scan_max, --bxid_length

Configuration of BIF producer

Example BIF producer configuration is here:

BIF producer configuration 
[Producer.BIF1]
EUDAQ_DC="dc2,dc3,bxidColl1"
ReadoutDelay=10
#TriggerInterval = 1
ConnectionFile = file://conf/bif_connections.xml
DeviceName = minitlu_bif
#InternalTriggerInterval = 500
InternalTriggerInterval = 0
UseIntDACValues = 0
# DACIntThreshold0 = 0x0
# DACIntThreshold1 = 0x0
# DACIntThreshold2 = 0x0
# DACIntThreshold3 = 0x0
#BIF registers falling edge
DACThreshold0 = -0.4
DACThreshold1 = -0.4
DACThreshold2 = -0.4
DACThreshold3 = -0.4
WriteRawOutput = 0
RawFileName = "./data/BifRaw/bifraw-run%06d"
DumpRawOutput = 0
#DumpCycleInfoLevel levels: 2=full info, 1=1 character (@), 0=none
DumpCycleInfoLevel = 1
#DumpTriggerInfoLevel levels: 2=full info, 1=1 character (.), 0=none
DumpTriggerInfoLevel = 1
DumpSummary = 1
#RawFileName without .raw extension
WriteRawFileNameTimestamp = 1
#FirstBxidOffsetBins = 196668
CountRocFrom = 1 #by default bif counts from 0
FirstBxidOffsetBins = 67658 # in BIF tics
BxidLengthNs = 4000
# WaitAfterStopSeconds setting has 1 s uncertainty!
WaitAfterStopSeconds = 0
#0x0100=PX2, 
TriggerMask = 0xFFFFFF00 # input3=0xFFFFFF00, input2=0xFFFFF0F0, input1=0xFFFFCCCC, input0=0xFFFFAAAA, any input=0xFFFFFFFE
#enable RJ45 calice ahcal input:
DUTMask = 0x1
EnableRecordData = 0xFF
#When RedirectInputFromFile is set, an existing raw raw data file is used as a fake input from detector
#RedirectInputFromFile="/data/201805/bifraw-run060939__22p05p2018__16p38p52.raw"
#EventBuildingMode selects what event building will be used and
#EventBuildingMode = "ROC" #Complete readout cycle
#EventBuildingMode = "TRIGGERS" #packets for individual triggers are used
EventBuildingMode = "BXID" #everything from same BXID will be packed together. Beware of input delays
#EventNumberingPreference chooses a main numbering method, when both are available (especially in BUILD_BXID_VALIDATE and TRIGGERID event building mode) 
#EventNumberingPreference = "TRIGGERID"
EventNumberingPreference = "TIMESTAMP"

The IP addres and communication mechanism are defined in a separate configuration files: bif_connections.xml. There are 2 possible ways of communication with the BIF: Through the "Control Hub", or via UDP protocol directly.

Values, that need adjustment:

  • FirstBxidOffsetBins
  • DACThreshold0, DACThreshold1, DACThreshold2, DACThreshold3

  • DumpCycleInfoLevel, DumpTriggerInfoLevel

BIF time offset

Similarly to the LDA time offset, The BIF has also its own offset constant, which needs to be added to the start-of-the-acquisition TimeStamp in order to get to the time when BXID 0 starts.

Easiest way is to use following script from the same project project: https://github.com/jkvas/AHCAL-RAWutils

script parameters 
$AHCAL_RAWUTIL_PATH/ahcalBifCorrelation/ahcalbifcorrelate \
--spiroc_raw_file $AHCAL_RAW_FILE \
--require_hitbit \
--shift_scan_max 500000 \
--shift_scan_method 1 \
--bif_raw_file $BIF_RAW_FILE \
--start_position -1 \
--bxid_length 5120 >BIF_offset.txt

The output file contains the optimal value with the number of hits:

#maximum correlation at: 451652 hits:6126703

It is highly recommended to check the graph of number of time correlations vs the offset. Output can be displayed with gnuplot command:

gnuplot # start the gnuplot
plot 'BIF_offset.txt' u 1:3 w lines #type this in gnuplot to get the plot

Typical results without validation look like this: (full range on the left and zoomed with right mouse button click on the right):

LED scan with EUDAQ

Preparation of configuration files

In /home/calice/eudaq/conf/  directory:
usually this is already there: LED_xxx 
if you need a new subdirectory: mkdir LED_long_comissioning; copy existing conf files to this new directory (if this is not prepared already).

In this directory (here: LED_long_commissioning): there are two important files:
shell scripts: short_LED.sh or long_LED.sh
config file: xxLED.conf (one file for both run lengths).

in the config file (if changes are needed, basic settings):

producer.ahcal1 section: 
fileLEDsettings has to point to existing path on Labview pc (has to be created there if it does not exist). here: D:\\LED_CONFIG\\GENERATE\\...
change IPaddress to IP of the Labview PC

producer.bif1 section:
FirstBXIDoffsetbins=xxx (offset of BIF timing for power pulsing switch on time, different for pp or no pp). See logbook for values.

that's it for config file.

in the shell script file long_LED.sh:
LABVFolder="LONG (Folder has to be on Labview PC).
NV="xx" number of voltages VCALIB
V0="yy" first voltage
DMV="zz" step width in mV
events="aa" number of readout cycles per voltage.
very end of file: scp command: change in this line ip-address and folder as in the conf file (values for Labview PC and folders there).

Run the script if any changes in configuration: 
./long_LED.sh
enter password of Labview PC when prompted

Now the new folder and ini files should be on the Labview PC. On the EUDAQ PC the new config files appear in the same folder.
LED_000.conf is for pedestal.
After last LED run there is the last run again with pedestal setting, so you get for n defined measurements n+1 config files.
In each config file, the confilg file for the next LED voltage is listed.
The last config file loops itself - neverending pedestal runs.

The ini files on the Labview PC show all voltages that will be measured.

Starting EUDAQ

Before starting EUDAQ, the Labview has to be running with all settings defined for power pulsing, temperature compensation, Spill, validation, write text files or not, correct sow-control files must be loaded to detector. Check enable section. In "Take Data" tab, define "Setting Lab mode" with trigger distance 100, 16 triggers. "Auto Listen" must be on! Define how often temperatures should be read.


On EUDAQ PC:
cd /home/calice/EUDAQ/
the relevant file is a shell script "STARTRUN_AHCAL_LEDxxx.sh"
In this file check what part of EUDAQ should be run:
- Run Control
- Log Collector
- Data Collector: Direct save data collector should be there.
- Producer "AHCAL" should be there. "BIF" if BIF is there and wanted.

start by ./STARTRUN_AHCAL...sh
EUDAQ windows open.
In The GUI, (bottom side), all the collectors should appear that we want.

Press load ini file (home/calice/eudaq/con/ahcal.ini). Press init.
All collectors and window change to "unconf".

Load Config file from the newly created directory. Choose LED_000.conf (first of the new created onfig files).
Press "Config".
Everything changes to "configured".

Press start.
Check in Labview if run starts properly. 
Labview does now HV adjustement when temp-compensation was defined. 
Automatic actions to be seen on tab "Take data" in main Labview.
Labview now starts with pedestal run (VCALIB=0, HV on).
check that "# elements in data queue" stays low and does not increase. If so, some setting is wrong, maybe trigger distance too large (stay below 100).

Result file (EUDAQ PC): /home/calice/EUDAQ/data/ there are up to 5 directories:
For this LED run, see directories: AHCALraw, BIFraw, EUDAQraw. Filenames also end with .raw.

Convert data to LCIO format

~/eudaq/bin/euCliConverter can convert the eudaq raw (not the ahcal raw raw) to slcio.
All data files to be converted should be in one folder alone. Move away old data files that don't have to be converted (again). Don't convert files that are still been written to by the measurement.

Goto /home/calice/eudaq/data/eudaqRaw/ folder.
Example convert command:  ls Run0620*LED* | parallel -j 3 ~/eudaq/bin/euCliConverter -i {} -o /mnt/hdd2/slcio/LED/20181006_long/{}.slcio
modify to the right run number ("Run0620*LED*") and target path ("20181006_long/"). Target path has to exist before starting the conversion.
Option "parellel -j 3" means number of processes that are started in parallel for the conversion. A smaller number

EUDAQ with telescope

Running with telescope requires a different CCC firmware, different labview ccc config file and different EUDAQ configuration, because following events have to be synchronized: AHCAL, BIF, TLU, Telescope and possibly DESY table. Details are described in Testbeam documentation in March 2019.

TODO move the documentation here.

EUDAQ installation on UBUNTU 18.04

needed software:

installation failed mainly due to the IPBUS / UHAL. TODO

Virtual machine (Virtualbox):

use CERN centos 7: http://linux.web.cern.ch/linux/centos7/docs/install.shtml. Too slow

use standard centos 7: https://www.centos.org/download/, full image

  • enable network
  • add user calice, make the user administrator
  • install additions

Pre-requirements:

Centos Repositoris:

sudo yum install epel-release

CACTUS https://ipbus.web.cern.ch/ipbus/doc/user/html/software/install/yum.html

sudo curl http://ipbus.web.cern.ch/ipbus/doc/user/html/_downloads/ipbus-sw.centos7.x86_64.repo -o /etc/yum.repos.d/ipbus-sw.repo
sudo yum clean all
sudo yum groupinstall uhal

Root: https://root.cern.ch/downloading-root

System environments (.bashrc):

source /home/calice/root/bin/thisroot.sh 
export LD_LIBRARY_PATH=/opt/cactus/lib:$LD_LIBRARY_PATH

Cmake:

sudo yum install cmake3 cmake3-gui

xterm:

sudo yum install xterm

zlib:

sudo yum install zlib-devel.x86_64

boost libraries

sudo yum install boost boost-thread boost-devel

QT5 libraries:

yum install qt5-qtbase-devel.x86_64

s

 /media/sf_virtual/data/logs

EUDAQ

git clone https://github.com/jkvas/eudaq.git eudaq-kvas
git checkout calice_ahcal_testbeam2019March 
mkdir build
cd build
cmake3-gui -DBOOST_SYSTEM_NO_DEPRECATED ../
#enable calice and ahcal
make 
make install

git clone https://github.com/jkvas/eudaq.git eudaq-kvas


  • No labels