MPI on Fedora 34
On recent Ubuntu distributions, the following works as expected:
$ ./waf --run third-distributed
This simulation requires 2 and only 2 logical processors.
Command ... exited with code 1
$ mpirun -np 2 third-distributed
'build' finished successfully (0.916s)
'build' finished successfully (0.923s)
However, on Fedora 34, I observe:
$ ./waf --run third-distributed
Command ['/home/buildslave/jenkins/workspace/daily-valgrind/label/Fedora-34/ns-3-allinone/ns-3-dev/build/src/mpi/examples/ns3-dev-third-distributed-debug'] terminated with signal SIGILL. Run it under a debugger to get more information (./waf --run <program> --gdb").
$ mpirun -np 2 third-distributed
Command ['/home/buildslave/jenkins/workspace/daily-valgrind/label/Fedora-34/ns-3-allinone/ns-3-dev/build/src/mpi/examples/ns3-dev-third-distributed-debug'] terminated with signal SIGILL. Run it under a debugger to get more information (./waf --run <program> --gdb").
Under gdb, it says:
Thread 1 "ns3-dev-third-d" received signal SIGILL, Illegal instruction.
0x00007fffef717590 in fi_psm3_ini () from /lib64/libfabric.so.1
(gdb) bt
(gdb) bt
#0 0x00007fffef717590 in fi_psm3_ini () from /lib64/libfabric.so.1
#1 0x00007fffef60f7ae in fi_ini () from /lib64/libfabric.so.1
#2 0x00007fffef612dcd in fi_getinfo () from /lib64/libfabric.so.1
#3 0x00007ffff410a5ce in mca_btl_ofi_component_init ()
from /usr/lib64/openmpi/lib/openmpi/mca_btl_ofi.so
#4 0x00007ffff549f972 in mca_btl_base_select ()
from /usr/lib64/openmpi/lib/libopen-pal.so.40
#5 0x00007ffff4114188 in mca_bml_r2_component_init ()
from /usr/lib64/openmpi/lib/openmpi/mca_bml_r2.so
#6 0x00007ffff5badd74 in mca_bml_base_init ()
from /usr/lib64/openmpi/lib/libmpi.so.40
#7 0x00007ffff5bedb24 in ompi_mpi_init ()
from /usr/lib64/openmpi/lib/libmpi.so.40
#8 0x00007ffff5b8d582 in PMPI_Init () from /usr/lib64/openmpi/lib/libmpi.so.40
#9 0x00007ffff7f05b97 in ns3::GrantedTimeWindowMpiInterface::Enable (
this=0x535130, pargc=0x7fffffffb70c, pargv=0x7fffffffb700)
at ../src/mpi/model/granted-time-window-mpi-interface.cc:168
#10 0x00007ffff7f194f2 in ns3::MpiInterface::Enable (pargc=0x7fffffffb70c,
pargv=0x7fffffffb700) at ../src/mpi/model/mpi-interface.cc:120
#11 0x000000000041cfdd in main (argc=1, argv=0x7fffffffd498)
at ../src/mpi/examples/third-distributed.cc:110
Version info:
[buildslave@fedora-34 ns-3-dev]$ ompi_info
Package: Open MPI
mockbuild@buildvm-x86-09.iad2.fedoraproject.org
Distribution
Open MPI: 4.1.0
Open MPI repo revision: v4.1.0