Update intel mpi jobs to work around recent shared memory related CI failure
Description
Workaround for failing intel mpi toolchain (that started recently), e.g. https://gitlab.com/octopus-code/octopus/-/jobs/9714703681
Octopus crashes with
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 0 PID 115549 RUNNING AT mpsd-hpc-ibm-023
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================
----------------------------------------
These are the last lines of stderr:
----------------------------------------
Assertion failed in file ../../src/mpid/ch4/shm/posix/eager/include/intel_transport_recv.h at line 1160: cma_read_nbytes == size
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(MPL_backtrace_show+0x1c) [0x1509f7c7236c]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(MPIR_Assert_fail+0x21) [0x1509f7629131]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x920d5c) [0x1509f7920d5c]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x91e33b) [0x1509f791e33b]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x91bbae) [0x1509f791bbae]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x2f58b4) [0x1509f72f58b4]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x87a149) [0x1509f787a149]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x4a02fe) [0x1509f74a02fe]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x12d53d) [0x1509f712d53d]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x2364a6) [0x1509f72364a6]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x212453) [0x1509f7212453]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(+0x367129) [0x1509f7367129]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/release/libmpi.so.12(MPI_Alltoall+0x5a8) [0x1509f7176e18]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/libmpifort.so.12(+0xd4734) [0x1509f8cd4734]
/opt_mpsd/linux-debian12/debug-impi-failure/sandybridge/spack/opt/spack/linux-debian12-sandybridge/intel-2021.10.0/intel-oneapi-mpi-2021.10.0-s6psu7cb73m7dxnxriiu4ee3f5w2msr7/mpi/2021.10.0/lib/libmpifort.so.12(mpi_alltoall_f08ts_+0x20f) [0x1509f8c4916f]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x1d2fd31]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x12d069e]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x12d299d]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x129b708]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x129e0a0]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0xc1e6ae]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x7a6988]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x5049b3]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x4e7b20]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x4bf704]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x4ab797]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x4280cd]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a) [0x1509f4c4624a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x1509f4c46305]
/scratch/cioctopu/static/9722740924/octopus-code/octopus/cmake-build-ci-intel-full-mpi/octopus() [0x427fe1]
Abort(1) on node 1: Internal error
The fix is based on https://github.com/oneapi-src/distributed-ranges/issues/612
Checklist
-
I have checked that my code follows the Octopus coding standards -
I have added tests for all the new features added in this request.
Edited by Martin Lang