Page tree

Info:

This page descripes how to perform the data post processing of the datasets taken with the "run_asaxs" script at P62:

  • standard SAXS/WAXS
  • anomalous SAXS/WAXS
  • in situ SAXS/WAXS (dataset which have been recorded with the  "run_asaxs" script)

P62_reduction python package:

location: ....

or

Get a copy of it from the DESY_STASH repository:   git clone https://<desy_username>@stash.desy.de/scm/saxsmat/p62_reductiontool.git


In a terminal/cmd line you can simply run the following to get the help information about the package.

input command line:   p62_reduction -h

output:


The whole processing chain is slitted into several steps. The order of post processing steps on the 1D curves is not fixed but I recommend to follow the order 1-10 as it will be descriebed below.


1st step - Generation of the calibration and mask files

The integration is done by the PyFAI package. Therefore, you have do create "poni files" and "mask files" for each detector and each sample to detector distance using the tool: pyfai-calib2

For the reduction package it is important that the poni file and mask file naming and location follows the following rules otherwise the package can not find them automatically.

Example of naming and location:

The raw data have been saved in the following directory  D:/Hercules_Course/raw/test_new

Than you have to create a subfolder called: configuration within this main folder:   D:/Hercules_Course/raw/test_new/configuration

Put the poni files and the mask file inside this configuration folder.

Naming of the poni files (SAXS):  config_saxs_sdetx_%%%.poni   ← the %%% has to be replaced with motor value of sdetx.

Naming of the mask files (SAXS):  mask_saxs_sdetx_%%%.edf   ← the %%% has to be replaced with motor value of sdetx.

Naming of the poni files (WAXS):  config_waxs_sdetx_%%%.poni   ← the %%% has to be replaced with motor value of sdetx.

Naming of the mask files (WAXS):  mask_waxs_sdetx_%%%.edf   ← the %%% has to be replaced with motor value of sdetx.

Info: If you do not know the sdetx value of the measurment you can use the programm: "silx view" to open one nexus file and search for the entry: /scan/data/motor/sdetx


For the example the content of the configuration files looks like:

D:/Hercules_Course/raw/test_new/configuration

                                      |--  config_saxs_sdetx_2000.poni

                                      |--  config_waxs_sdetx_2000.poni

                                      |--  mask_saxs_sdetx_2000.edf

                                      \--  mask_waxs_sdetx_2000.edf

Important for anomlous SAXS/WAXS datasets:

Configuration files are needed only for one energy. The programm will automatically change the waelength for the processing later.

It turned out that this procedure works much better if you force the SAXS-constrains in the pyfai-calib2 programm. Meaning force all three rotation to be zero.


2nd step - Azimuthal integration of 2D patterns

You have two options to perform the azimuthal integration:

The option -int_cores <number>   allows specifiing the number of parallel cores. Default: 4 cores. The SAXS and WAXS integration gets individual cores as well. Meaning in total 8 cores are used if you specify -int_cores 4.  

Integrate all frames that are available in the directory

>> python3/run p62_reduction.py -i <path of raw data> -int -int_cores <number of cores, default: 4>

e.g.: python3 p62_reduction.py -i D:/Hercules_Course/raw/test_new -int 


Integrate all frames that are available in the directory but skip already prevously integrate once

>> python3/run p62_reduction.py -i <path of raw data> -int -int_skip

e.g.: python3 p62_reduction.py -i D:/Hercules_Course/raw/test_new -int -int_skip


The azimuthal integrated 1D curves will be saved in the following folders: 

e..g: D:/Hercules_Course/processed/test_new/                       ← it is the same as the raw data directory only that the raw subfolder is changed to processed ← DESY convention.

                                                 |-- uncorrected_saxs                  ← This subfolder contains all the 1D SAXS curves as single nexus file.

                                                 \-- uncorrected_waxs                 ← This subfolder contains all the 1D WAXS curves as single nexus file.

Important: These 1D curves are only aziumthally integrated. No correction have been made. The nexus files also contains all stored metadata of the experiment.


3nd step - Generation of a single nexus file with all 1D curves

After the integration of the 2D images to 1D curves all single 1D curves have to be combined in a single NEXUS file. This is easier to handle for the next processing steps and it is faster for data transfer.

Do merge all the single 1D curves into a single NEXUS file you can use the following commands. The outcome of this tool is a single nexus files saved in the processed folder.


>> python3/run p62_reduction.py -i <path of raw data> -create_nexus 

e.g.: python3 p62_reduction.py -i D:/Hercules_Course/raw/test_new -create_nexus 

the output file generated is:  D:/Hercules_Course/processed/test_new/test_new_v1.nxs

or

>> python3/run p62_reduction.py -i <path of raw data> -create_nexus -v %%%     <- replace %%% with a version number of your processing e.g. 2

e.g.: python3 p62_reduction.py -i D:/Hercules_Course/raw/test_new -create_nexus -v 2

the output file generated is:  D:/Hercules_Course/processed/test_new/test_new_v2.nxs


The version number allows testing different post processing.

You can use the "silx view" programm to look to the stored data within this NEXUS file. The data are sorted by the sample name.

The output data arrays are called: "uncorrected_saxs" / "uncorrected_waxs"

Screenshot:

The data arrays is normally 3 dimensional:  energy_index X repeat_index X q_index                   ← in the example we measured at 6 energies and each time 50 repeats. SAXS by default uses 1000 q values and WAXS uses 2000 q values.


4th step - Normalization of 1D curves

>> python3/run p62_reduction.py -i <path of data (can be raw or processed)> -v <version> -norm -norm_vers <version> -norm_input <input data array>

The -v options allows choosing the current process verion default is 1

In the moment three different ways of normalization are implemented:

  1. By transmission and I0 intensity                 -norm_vers 0
  2. By transmitted intensity I1 (beam stop) only     -norm_vers 1
  3. By aquisition time                               -norm_vers 2

The -norm_input option allows choosing the input data array. By default it is the "uncorrected" data arrays.  The programm will automatically add "_saxs" / "_waxs" to the array name.

Default: (by transmission and I0, input data: "uncorrected")

>> python3/run p62_reduction.py -i <path of data (can be raw or processed)> -norm 

In-situ: (by I1, input data: "uncorrected")

>> python3/run p62_reduction.py -i <path of data (can be raw or processed)> -norm -norm_vers 1

In-situ but no beamstop value: (by time, input data: "uncorrected")

>> python3/run p62_reduction.py -i <path of data (can be raw or processed)> -norm -norm_vers 2


The output data arrays are called: "normalized_saxs" / "normalized_waxs"


Screenshot:

The corresponding datasets have a few attributes that shows the norm_version and the input dataset name.


5th step - Background Subtraction of 1D curves

>> python3/run p62_reduction.py -i <path of data (can be raw or processed)> -v <version> -bkg -bkg_factor <scale factor> -bkg_list <dict of background names> -bkg_input <array name>

The -v options allows choosing the current process verion default is 1

The -bkg_input option allows choosing the input data array. By default it is the "normalized" data arrays.  The programm will automatically add "_saxs" / "_waxs" to the array name.

The -bkg_factor <value>  Includes a scaling factor for the background intensity. Default: 1.0

The -bkg_list <dict>  Allows changing the background sample name for given samples. The structure of the dict is as follows:  'sample_name':'background_name'.

                                 e.g:  bkg_list "{'glassy_carbon':'empty_hole', 'Au5nm':'kapton'}"       <- this will change the background for glassy_carbon to empty_hole and the Au5nm background to kapton.

The output data arrays are called: "background_corrected_saxs" / "background_corrected_waxs"

Screenshot:

6th step - Absolute scattering intensity calibration of 1D curves

>> python3/run p62_reduction.py -i <path of data (can be raw or processed)> -v <version> -abs -abs_relative_sample <sample name of glassy carbon> -abs_reference_file <reference file> -abs_input <array name> -abs_int_range_qmin <qmin> -abs_int_range_qmax <qmax>

The -v options allows choosing the current process verion default is 1

The -abs_input option allows choosing the input data array. By default it is the "background_corrected" data arrays.  The programm will automatically add "_saxs" / "_waxs" to the array name.

The -abs_int_range_qmin <value>  Qmin value of the integral region for determining the scale factor. Default: 0.6 nm^1

The -abs_int_range_qmax <value>  Qmax value of the integral region for determining the scale factor. Default: 1.0 nm^1

The -abs_relative_sample <sample_name>  Sample name of the reference sample. Default: glassycarbon

The -abs_reference_file <path + filename>  Path and filename of the reference intensity file. Default: p62_glassycarbon.dat


The output data arrays are called: "absolute_scaled_saxs" / "absolute_scaled_waxs"

Screenshot:

The scale factors are saved within the absolute_scaled_saxs group as new array called: scalefactor

7th step - Modification of Q-range of 1D curves


Screenshot:

8th step - Averaging repeated samples of 1D curves


Screenshot:


9th step - Set final 1D curves


Screenshot:

10th step - Export of 1D curves as ASCII files


Screenshot:


P62_view

  • No labels