- Sources: https://github.com/ECP-WarpX/WarpX
- Documentation: https://ecp-warpx.github.io/
- License: https://github.com/ECP-WarpX/WarpX/blob/development/LICENSE.txt
Setting warpx environment
warpx modules do not provide a real warpx-binary, but only set the environment enabling warpx compilation!
@max-wgse001:~$ module load maxwell warpx Loading modules: gcc/9.3 openmpi/4.0.4 hdf5/1.10.6 Adios2: /software/gcc/9.3.0/openmpi/4.0.4/adios2/2.6.0 openPMD: /software/gcc/9.3.0/openmpi/4.0.4/openPMD-api/0.12.0 warpX: /software/gcc/9.3.0/openmpi/4.0.4/warpx_directory - sample installation Additional variables defined: ADIOS2: /software/gcc/9.3.0/openmpi/4.0.4/adios2/2.6.0 openPMD: /software/gcc/9.3.0/openmpi/4.0.4/openPMD-api/0.12.0 Building warpx: should work out of the box: module load naxwell warpx; make USE_MPI=TRUE USE_OPENPMD=TRUE with GPU support: module load naxwell warpx cuda/11.0; make USE_MPI=TRUE USE_OPENPMD=TRUE USE_GPU=TRUE openpmd backends supported: bp,h5 (adios2,hdf5) # On nodes equipped with a GPU you could in addition enable CUDA @max-wgse001:~$ module load cuda/11.0
Building warpx
# environment module load maxwell warpx module load cuda/11.0 # get the code and dependencies cd /scratch/me/ rm -rf warpx_directory/ mkdir warpx_directory cd warpx_directory/ git clone https://github.com/ECP-WarpX/WarpX.git git clone https://github.com/ECP-WarpX/picsar.git git clone --branch development https://github.com/AMReX-Codes/amrex.git cd WarpX # build in place make USE_MPI=TRUE USE_OPENPMD=TRUE USE_GPU=TRUE # USE_GPU=FALSE on non-gpu nodes # worked without errors both for CPU and GPU modes
Testing the backends
# get sample input file cp /software/gcc/9.3.0/openmpi/4.0.4/warpx_directory/WarpX/Bin/inputs_3d . # test the hdf5 backend perl -pi -e 's|diag1.openpmd_backend =.*|diag1.openpmd_backend = h5|' inputs_3d mpirun -np 4 /scratch/warpx_directory/me/WarpX/Bin/main3d.gnu.TPROF.MPI.CUDA.OPMD.ex inputs_3d # test the bp backend perl -pi -e 's|diag1.openpmd_backend =.*|diag1.openpmd_backend = bp|' inputs_3d mpirun -np 4 /scratch/warpx_directory/me/WarpX/Bin/main3d.gnu.TPROF.MPI.CUDA.OPMD.ex inputs_3d # output generated ls -lrt diags/diag1/ -rw-r--r-- 1 me it 169496112 Sep 15 12:14 openpmd_000000.h5 -rw-r--r-- 1 me it 225484720 Sep 15 12:14 openpmd_001000.h5 drwxr-xr-x 2 me it 4096 Sep 15 12:23 openpmd_000000.bp drwxr-xr-x 2 me it 4096 Sep 15 12:23 openpmd_001000.bp