Summary

Apache Spark™ is a unified analytics engine for large-scale data processing.

Using  apache spark

The installations are simple dumps of spark-<version>-bin-hadoop<version>.tgz. A very primitive sample spark-batch-script can be found under /software/spark/slurm/slurm.spark.sh.

[max]% module load maxwell spark