Summary
Source: https://github.com/mark-glass/comsyl
License: MIT License
Path: /software/comsyl
Documentation: https://github.com/mark-glass/comsyl/wiki
COMSYL makes the coherent mode decomposition of synchrotron radiation emitted by electrons passing through an undulator placed in a storage ring. COMSYL permits naturally the statistical analysis and propagation of the cross spectral density along the beamline optics. The coherence properties of the X-ray beam at any point of the beamline are completely given in terms of the eigenvalues and coherent modes of the cross spectral density (copied from https://github.com/mark-glass/comsyl ).
Using comsyl
comsyl is available as a standalone installation, compiled with openmpi
[max]% module load maxwell comsyl [max]% mpirun -np 4 python /software/comsyl/comsyl1env/comsyl/comsyl/calculateAutocorrelation.py /software/comsyl/comsyl1env/comsyl/comsyl/configurations/septest_cm_new_u18_2m_1h.json
Sample batch script
/software/comsyl/sbatch-comsyl.sh
#!/bin/bash
#SBATCH --partition=all
#SBATCH --time=2-00:00:00
#SBATCH --job-name=comsyl
#SBATCH --output=sbatch-comsyl.out
#SBATCH --nodes=1
unset LD_PRELOAD
source /etc/profile.d/modules.sh
module purge
module load maxwell comsyl
mkdir -p calculations # output directory used by samples
# sample 1: runs for a few minutes
time mpirun -np 4 --mca pml ucx python /software/comsyl/comsyl1env/comsyl/comsyl/calculateAutocorrelation.py \
/software/comsyl/comsyl1env/comsyl/comsyl/configurations/septest_cm_new_u18_2m_1h.json
# sample 2: runs for several hours. this sample required petsc configuration --with-64-bit-indices
mpirun -N $(( $(nproc) / 2)) --mca pml ucx python /software/comsyl/comsyl1env/comsyl/comsyl/calculateAutocorrelation.py \
/software/comsyl/comsyl1env/comsyl/comsyl/configurations/high_flux/hf_new_u18_1m_3h.json
Building comsyl
Installation instructions for comsyl can be found at https://github.com/mark-glass/comsyl/wiki/Install-COMSYL. We had to deviate a bit from the proposed procedure:
# # base environment # instdir=/software/comsyl scrdir=/scratch/$USER mkdir -p $instdir $scrdir cd $instdir virtualenv-3.6 -p python3 --system-site-packages comsyl1env source $instdir/comsyl1env/bin/activate python -m pip install --upgrade pip # *** changed: # petsc uses a bit of deprecated code, so downgrade pip install numpy==1.18.4 module load mpi/opnempi-x86_64 export PYTHONPATH=/usr/lib64/python3.6/site-packages/openmpi # # srxraylib # cd $scrdir git clone https://github.com/lucarebuffi/srxraylib cd srxraylib python setup.py build pip install -e . --no-binary :all: cd $scrdir rm -rf srxraylib # # syned # cd .. git clone https://github.com/lucarebuffi/syned cd syned python setup.py build pip install -e . --no-binary :all: # if fails use instead: python setup.py develop cd $scrdir rm -rf syned # # comsyl # cd $instdir git clone https://github.com/mark-glass/comsyl cd comsyl/ python setup.py build python setup.py develop # *** changed: # just copy the python modules into PYTHONPATH cp -riv comsyl comsyl.egg-info ../lib/python3.6/site-packages/ # *** changed: patch comsyl # numpy linspace expects an integer for the number of evenly spaced samples # dim_x * resample_x is however a float which non-ancient numpy versions won't accept # diff $instdir/comsyl1env/lib/python3.6/site-packages/comsyl/waveoptics/Wavefront.py $instdir/comsyl1env/lib/python3.6/site-packages/comsyl/waveoptics/Wavefront.py~ 489c489 < new_x = np.linspace(x_min, x_max, int(dim_x * resample_x)) --- > new_x = np.linspace(x_min, x_max, dim_x * resample_x) 503c503 < new_y = np.linspace(y_min, y_max, int(dim_y * resample_y)) --- > new_y = np.linspace(y_min, y_max, dim_y * resample_y) # # *** changed: petsc/slepc # the combination of petsc, slepc from the installation instruction is not valid # versions of petsc, slepc should match # - use a recent version of petsc, slepc # - add --with-petsc4py=1, also build the python module # - add --with-64-bit-indices, some of the samples won't run without # - don't optimize for hardware cd $instdir wget https://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.15.2.tar.gz tar xf $scrdir/petsc-3.15.2.tar.gz cd petsc-3.15.2/ export PETSC_DIR=`pwd` export PETSC_ARCH=arch-linux2-c-opt ./configure --with-mpi-f90=/usr/bin/mpif90.openmpi --with-mpi-dir=/usr/lib64/openmpi --with-scalar-type=complex --with-debug=0 COPTFLAGS='-O3' CXXOPTFLAGS='-O3' FOPTFLAGS='-O3' --with-petsc4py=1 --with-64-bit-indices make -j 8 export PYTHONPATH=$instdir/petsc-3.15.2/arch-linux2-c-opt/lib:$PYTHONPATH # # slepc # cd $instdir wget https://slepc.upv.es/download/distrib/slepc-3.15.1.tar.gz tar xf slepc-3.15.1.tar.gz cd slepc-3.15.1/ export SLEPC_DIR=`pwd` make SLEPC_DIR=$instdir/slepc-3.15.1 PETSC_DIR=$instdir/petsc-3.15.2 PETSC_ARCH=arch-linux2-c-opt # # srw ... use install from comsyl # - nothing done