Maxwell : puffin

Puffin (Parallel Unaveraged Fel INtegrator) simulates a Free Electron Laser (FEL). Puffin is a massively parallel numerical solver for an unaveraged, 3D FEL system of equations, and is written in Fortran 90, using MPI and OpenMP.

Using puffin:

Simply load the corresponding module:

module load maxwell puffin
which puffin
   /software/puffin/1.9.0.2/bin/puffin 

# the module also defines some environment variables to simplify the commands. Simply unset if disturbing:
OMPI_MCA_pml=ucx
OMPI_MCA_opal_warn_on_missing_libcuda=0
UCX_LOG_LEVEL=error

Running puffin batch-jobs on Maxwell

puffin is a hybrid OPENMP/MPI application. As it turns out, performance is best when using a small number of MPI processes (4) in favor of OPENMP threads.

A sample batch script:

#!/bin/bash 
#SBATCH --partition=upex
#SBATCH --time=4-00:00:00
#SBATCH --nodes=4
#SBATCH --constraint='[7542|Gold-6140|Gold-6240]'
#SBATCH --output=puffin-%j.out
#SBATCH --job-name=puffin
unset LD_PRELOAD
source /etc/profile.d/modules.sh

module load maxwell puffin

mpirun -N 4 puffin puffin.in  

Simple benchmark

a non-representative test-run indicates, that a number of 4 MPI processes per node is most efficient - unless running an unrealistic small number of periods.