petsc4py: poisson3d and wrap_f2py demos fail with segfault or Python int too large to convert to C int
petsc4py provides a number of demos. The poisson3d and wrap_f2py demos consistently fail on my system (debian unstable). This is with PETSc 3.18.3, gcc-12, python3.10 and python3.11, numpy 1.23.
Full test logs can be found at https://ci.debian.net/packages/p/petsc4py/unstable/amd64/ or https://ci.debian.net/packages/p/petsc4py/testing/amd64/ . The current most recent is https://ci.debian.net/data/autopkgtest/unstable/amd64/p/petsc4py/30112613/log.gz Debian CI tests run in 4 parts: unit tests (C++ and python) and demos for the standard (32 bit) and 64-bit builds. It's the python demos ("run-demos") where the failure occurs
In most cases a segfault occurs. But on debian/testing (with python3.10) wrap_f2py reports
OverflowError: Python int too large to convert to C intFor poisson3d, the error is
make -C poisson3d
make[1]: Entering directory '/tmp/autopkgtest-lxc.9i0ww8sl/downtmp/build.0wL/src/test-demos/poisson3d'
f2py3 --quiet --noarch --f90flags='' -DF2PY_REPORT_ON_ARRAY_COPY=1 -c del2lib.f90 -m del2lib
WARN: Could not locate executable armflang
python3 poisson3d.py
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see https://petsc.org/release/faq/#valgrind and https://petsc.org/release/faq/
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run
[0]PETSC ERROR: to get more information on the crash.
[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 59.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
make[1]: Leaving directory '/tmp/autopkgtest-lxc.9i0ww8sl/downtmp/build.0wL/src/test-demos/poisson3d'I can reproduce locally and get a backtrace (make; gdb python3, run poisson3d.py)
Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00000000006298a0 in new_threadstate (interp=0x0) at ../Python/pystate.c:801
801 ../Python/pystate.c: No such file or directory.
(gdb) bt
#0 0x00000000006298a0 in new_threadstate (interp=0x0) at ../Python/pystate.c:801
#1 0x0000000000629866 in PyThreadState_New (interp=<optimized out>) at ../Python/pystate.c:855
#2 0x0000000000460ac7 in PyGILState_Ensure () at ../Python/pystate.c:1685
#3 0x00007ffff75149a1 in ?? () from /usr/lib/petsc/lib/python3/dist-packages/petsc4py/lib/PETSc.cpython-311-x86_64-linux-gnu.so
#4 0x00007ffff62be88c in MatDestroy () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#5 0x00007ffff675c822 in PCReset () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#6 0x00007ffff675c875 in PCDestroy () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#7 0x00007ffff685b95c in KSPDestroy () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#8 0x00007ffff611a930 in PetscObjectDestroy () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#9 0x00007ffff6131cab in PetscGarbageCleanup () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#10 0x00007ffff6129a46 in PetscFinalize () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#11 0x00007ffff72b50c0 in ?? () from /usr/lib/petsc/lib/python3/dist-packages/petsc4py/lib/PETSc.cpython-311-x86_64-linux-gnu.so
#12 0x0000000000458b8e in call_ll_exitfuncs (runtime=<optimized out>) at ../Python/pylifecycle.c:2930
#13 Py_FinalizeEx () at ../Python/pylifecycle.c:1929
#14 0x0000000000650294 in Py_RunMain () at ../Modules/main.c:682
#15 0x00000000006248b7 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:734
#16 0x00007ffff7c8f18a in __libc_start_call_main (main=main@entry=0x624820 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffe008) at ../sysdeps/nptl/libc_start_call_main.h:58
#17 0x00007ffff7c8f245 in __libc_start_main_impl (main=0x624820 <main>, argc=2, argv=0x7fffffffe008, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdff8)
at ../csu/libc-start.c:381
#18 0x0000000000624751 in _start ()
For wrap_f2py, the error on debian/unstable is superficially the same,
python3 run_demo.py
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see https://petsc.org/release/faq/#valgrind and https://petsc.org/release/faq/
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run
[0]PETSC ERROR: to get more information on the crash.
[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 59.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
make[1]: *** [makefile:18: run] Error 59
make[1]: Leaving directory '/tmp/autopkgtest-lxc.9i0ww8sl/downtmp/build.0wL/src/test-demos/wrap-f2py'But the log on debian testing (with python3.10) is slightly different, see https://ci.debian.net/data/autopkgtest/testing/amd64/p/petsc4py/30110804/log.gz ,
python3 run_demo.py
Traceback (most recent call last):
File "/tmp/autopkgtest-lxc.ydc0dwi_/downtmp/build.EQx/src/test-demos/wrap-f2py/run_demo.py", line 58, in <module>
app.formInitGuess(snes, X)
File "/tmp/autopkgtest-lxc.ydc0dwi_/downtmp/build.EQx/src/test-demos/wrap-f2py/run_demo.py", line 18, in formInitGuess
ierr = Bratu2D.FormInitGuess(da, vec_X, self.lambda_)
OverflowError: Python int too large to convert to C int
make[1]: Leaving directory '/tmp/autopkgtest-lxc.ydc0dwi_/downtmp/build.EQx/src/test-demos/wrap-f2py'A backtrace is
Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007ffff6711fce in DMDAGetInfo () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
(gdb) bt
#0 0x00007ffff6711fce in DMDAGetInfo () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#1 0x00007ffff6744d0d in dmdagetinfo_ () from /usr/lib/x86_64-linux-gnu/libpetsc_real.so.3.18
#2 0x00007fffdd852708 in bratu2d::getgridinfo (da=..., grd=..., ierr=0) at ./Bratu2D.F90:40
#3 0x00007fffdd8528af in forminitguess (da=..., x=..., lambda=6, ierr=0) at ./Bratu2D.F90:191
#4 0x00007fffdd84eec6 in f2py_rout_Bratu2D_FormInitGuess (capi_self=<optimized out>, capi_args=<optimized out>, capi_keywds=<optimized out>, f2py_func=0x7fffdd852830 <forminitguess>) at ./Bratu2Dmodule.c:259
#5 0x0000000000516ab3 in _PyObject_MakeTpCall (tstate=0xa85258 <_PyRuntime+166328>, callable=<fortran at remote 0x7fffb227ced0>, args=<optimized out>, nargs=3, keywords=0x0) at ../Objects/call.c:214
#6 0x000000000052a570 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:4772
#7 0x000000000052222b in _PyEval_EvalFrame (throwflag=0, frame=0x7ffff7f77020, tstate=0xa85258 <_PyRuntime+166328>) at ../Include/internal/pycore_ceval.h:73
#8 _PyEval_Vector (args=0x0, argcount=0, kwnames=0x0, locals=<optimized out>, func=0x7ffff7c004a0, tstate=0xa85258 <_PyRuntime+166328>) at ../Python/ceval.c:6435
#9 PyEval_EvalCode (co=<code at remote 0xb363a0>, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:1154
#10 0x0000000000647f07 in run_eval_code_obj (tstate=0xa85258 <_PyRuntime+166328>, co=0xb363a0,
globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py') at remote 0x7ffff7c5b390>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7bf0ae0>, '__file__': '/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', '__cached__': None, 'sys': <module at remote 0x7ffff7be2ca0>, 'petsc4py': <module at remote 0x7ffff7617ec0>, 'PETSc': <module at remote 0x7ffff76594e0>, 'Bratu2D': <module at remote 0x7fffb22a0720>, 'App': <type at remote 0x12b9560>, 'OptDB': <petsc4py.PETSc.Options at remote 0x7fffb227ce80>, 'N': 16, 'lambda_': <float at remote 0x7ffff7799870>, 'do_plot': False, 'da': <petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, 'app': <App(da=<petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, lambda_=<float at remote 0x7ffff7799870>) at remote 0x7fffb229e0d0>, 'snes': <petsc4py.PETSc.SNES at remote 0x7fffb22a0950>, 'F': <petsc4py.PETSc.Vec a...(truncated),
locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py') at remote 0x7ffff7c5b390>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7bf0ae0>, '__file__': '/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', '__cached__': None, 'sys': <module at remote 0x7ffff7be2ca0>, 'petsc4py': <module at remote 0x7ffff7617ec0>, 'PETSc': <module at remote 0x7ffff76594e0>, 'Bratu2D': <module at remote 0x7fffb22a0720>, 'App': <type at remote 0x12b9560>, 'OptDB': <petsc4py.PETSc.Options at remote 0x7fffb227ce80>, 'N': 16, 'lambda_': <float at remote 0x7ffff7799870>, 'do_plot': False, 'da': <petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, 'app': <App(da=<petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, lambda_=<float at remote 0x7ffff7799870>) at remote 0x7fffb229e0d0>, 'snes': <petsc4py.PETSc.SNES at remote 0x7fffb22a0950>, 'F': <petsc4py.PETSc.Vec a...(truncated))
at ../Python/pythonrun.c:1714
#11 0x00000000006457cf in run_mod (mod=<optimized out>, filename=<optimized out>,
globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py') at remote 0x7ffff7c5b390>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7bf0ae0>, '__file__': '/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', '__cached__': None, 'sys': <module at remote 0x7ffff7be2ca0>, 'petsc4py': <module at remote 0x7ffff7617ec0>, 'PETSc': <module at remote 0x7ffff76594e0>, 'Bratu2D': <module at remote 0x7fffb22a0720>, 'App': <type at remote 0x12b9560>, 'OptDB': <petsc4py.PETSc.Options at remote 0x7fffb227ce80>, 'N': 16, 'lambda_': <float at remote 0x7ffff7799870>, 'do_plot': False, 'da': <petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, 'app': <App(da=<petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, lambda_=<float at remote 0x7ffff7799870>) at remote 0x7fffb229e0d0>, 'snes': <petsc4py.PETSc.SNES at remote 0x7fffb22a0950>, 'F': <petsc4py.PETSc.Vec a...(truncated),
locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py') at remote 0x7ffff7c5b390>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7bf0ae0>, '__file__': '/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', '__cached__': None, 'sys': <module at remote 0x7ffff7be2ca0>, 'petsc4py': <module at remote 0x7ffff7617ec0>, 'PETSc': <module at remote 0x7ffff76594e0>, 'Bratu2D': <module at remote 0x7fffb22a0720>, 'App': <type at remote 0x12b9560>, 'OptDB': <petsc4py.PETSc.Options at remote 0x7fffb227ce80>, 'N': 16, 'lambda_': <float at remote 0x7ffff7799870>, 'do_plot': False, 'da': <petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, 'app': <App(da=<petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, lambda_=<float at remote 0x7ffff7799870>) at remote 0x7fffb229e0d0>, 'snes': <petsc4py.PETSc.SNES at remote 0x7fffb22a0950>, 'F': <petsc4py.PETSc.Vec a...(truncated),
flags=<optimized out>, arena=<optimized out>) at ../Python/pythonrun.c:1735
#12 0x0000000000651920 in pyrun_file (fp=fp@entry=0xaf1220, filename=filename@entry='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', start=start@entry=257,
globals=globals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py') at remote 0x7ffff7c5b390>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7bf0ae0>, '__file__': '/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', '__cached__': None, 'sys': <module at remote 0x7ffff7be2ca0>, 'petsc4py': <module at remote 0x7ffff7617ec0>, 'PETSc': <module at remote 0x7ffff76594e0>, 'Bratu2D': <module at remote 0x7fffb22a0720>, 'App': <type at remote 0x12b9560>, 'OptDB': <petsc4py.PETSc.Options at remote 0x7fffb227ce80>, 'N': 16, 'lambda_': <float at remote 0x7ffff7799870>, 'do_plot': False, 'da': <petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, 'app': <App(da=<petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, lambda_=<float at remote 0x7ffff7799870>) at remote 0x7fffb229e0d0>, 'snes': <petsc4py.PETSc.SNES at remote 0x7fffb22a0950>, 'F': <petsc4py.PETSc.Vec a...(truncated),
locals=locals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py') at remote 0x7ffff7c5b390>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7bf0ae0>, '__file__': '/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', '__cached__': None, 'sys': <module at remote 0x7ffff7be2ca0>, 'petsc4py': <module at remote 0x7ffff7617ec0>, 'PETSc': <module at remote 0x7ffff76594e0>, 'Bratu2D': <module at remote 0x7fffb22a0720>, 'App': <type at remote 0x12b9560>, 'OptDB': <petsc4py.PETSc.Options at remote 0x7fffb227ce80>, 'N': 16, 'lambda_': <float at remote 0x7ffff7799870>, 'do_plot': False, 'da': <petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, 'app': <App(da=<petsc4py.PETSc.DMDA at remote 0x7fffb22a0680>, lambda_=<float at remote 0x7ffff7799870>) at remote 0x7fffb229e0d0>, 'snes': <petsc4py.PETSc.SNES at remote 0x7fffb22a0950>, 'F': <petsc4py.PETSc.Vec a...(truncated), closeit=closeit@entry=1, flags=0x7fffffffddd8) at ../Python/pythonrun.c:1630
#13 0x000000000065166b in _PyRun_SimpleFileObject (fp=0xaf1220, filename='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', closeit=1, flags=0x7fffffffddd8) at ../Python/pythonrun.c:440
#14 0x0000000000651494 in _PyRun_AnyFileObject (fp=0xaf1220, filename='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', closeit=1, flags=0x7fffffffddd8) at ../Python/pythonrun.c:79
#15 0x000000000065022f in pymain_run_file_obj (skip_source_first_line=0, filename='/home/drew/projects/petsc/build/petsc4py/demo/wrap-f2py/run_demo.py', program_name='/usr/bin/python3') at ../Modules/main.c:360
#16 pymain_run_file (config=0xa6b2a0 <_PyRuntime+59904>) at ../Modules/main.c:379
#17 pymain_run_python (exitcode=0x7fffffffddd4) at ../Modules/main.c:601
#18 Py_RunMain () at ../Modules/main.c:680
#19 0x00000000006248b7 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:734
#20 0x00007ffff7c8f18a in __libc_start_call_main (main=main@entry=0x624820 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffe008) at ../sysdeps/nptl/libc_start_call_main.h:58
#21 0x00007ffff7c8f245 in __libc_start_main_impl (main=0x624820 <main>, argc=2, argv=0x7fffffffe008, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdff8)
at ../csu/libc-start.c:381
#22 0x0000000000624751 in _start ()I'll upload a new debian package skipping these tests, so they won't show in future test logs. I wanted to report the error before I do that.
Edited by Drew Parsons