The performance regression testsuite is thought to run different tests
(at the moment only unit tests) for a lot of different input files to
test the performance of the code for various parts of the parameter
space.
It can be called as make check-performance from the build dir, or also
using the oct-run_performance_testsuite.sh script from the installed
bin directory (no parameters needed).
Some parameters can be used to tweak the behavior (when setting them as
environment variables):
TESTS: by default all, but can also be a list of tests to run
SLURM: if set to true, salloc + srun is used to run the performance
regression testsuite through slurm
SALLOC: if set to false, do not use salloc for running make
NODES: number of nodes to run on if using slurm
TASKS: how many tasks are available to run in parallel - useful with and without slurm
NPROCS: number of processors to use for each test run -> number of jobs to run in parallel is then given bi TASKS/NPROCS
Example: make check-performance SLURM=true NODES=2 TASKS=64
If running each test in parallel, one can use make check-performance SLURM=true NODES=2 TASKS=64 NPROCS=2.
When using the performance regression testsuite on slurm system inside a sbatch job, one can use TESTS=batch_ops SLURM=true SALLOC=false NODES=1 TASKS=40 NPROCS=1 $OCTOPUS_HOME/bin/oct-run_performance_testsuite.sh (in this example, only the batch_ops tests are run).
Also, parameters of the tests can be overridden by specifying OPRT_parameter_name environment variables with valid YAML code defining the corresponding parameter value array as in the combinations.yaml files.
Sebastian Ohlmann authored