Big matrix multiplicaton crashes when linking against Xenomai 3

Submitted by Arturo Laurenzi

Assigned to Nobody

Link to original bugzilla bug (#1482)
Version: 3.3 (current stable)
Operating system: Linux

Description

Created attachment 800
Small self-contained example. Builds with cmake.

A bad crash happens when performing a multiplication between sufficiently large matrices. It happens only when

the multiplication is done inside a (POSIX) thread, so not inside the main()
the executable is compiled with Xenomai 3 compilation flags (as given by xeno-config --posix --cflags/--ldflags).

Such flags on my system are as follows:

CFLAGS: -I/usr/xenomai/include/cobalt -I/usr/xenomai/include -D_GNU_SOURCE -D_REENTRANT -D__COBALT__ -D__COBALT_WRAP__

LDFLAGS: -Wl,--no-as-needed -Wl,@/usr/xenomai/lib/cobalt.wrappers -Wl,@/usr/xenomai/lib/modechk.wrappers /usr/xenomai/lib/xenomai/bootstrap.o -Wl,--wrap=main -Wl,--dynamic-list=/usr/xenomai/lib/dynlist.ld -L/usr/xenomai/lib -lcobalt -lmodechk -lpthread -lrt

Compiler: GCC 5.4.0
OS: Ubuntu 16.04 + Xenomai 3 (Cobalt)
Eigen: 3.3.4

I am attaching a simple executable that show the issue (only on Xenomai 3, as specified before). More specifically, multiplying a [100x39] by a [39x100] succeeds, while [100x40] by [40x100] fails.

This is the backtrace:

Thread 3 "xeno3_tests" received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7ffff7ff4700 (LWP 19199)]  
0x000000000041bfa5 in Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>::run (rows=100, cols=100, depth=40, _lhs=0x7ffff00008c0,   
    lhsStride=100, _rhs=0x7ffff00085d0, rhsStride=40, _res=0x7ffff00102e0, resStride=100, alpha=1, blocking=..., info=0x0)  
    at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:163  
163	    ei_declare_aligned_stack_constructed_variable(RhsScalar, blockB, sizeB, blocking.blockB());  
(gdb) bt  
#0  0x000000000041bfa5 in Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>::run (rows=100, cols=100, depth=40, _lhs=0x7ffff00008c0,   
    lhsStride=100, _rhs=0x7ffff00085d0, rhsStride=40, _res=0x7ffff00102e0, resStride=100, alpha=1, blocking=..., info=0x0)  
    at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:163  
#1  0x000000000041b13e in Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >::operator() (this=0x7ffff7ff3b20, row=0, rows=100, col=0, cols=100, info=0x0) at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:226  
#2  0x000000000041ad8c in Eigen::internal::parallelize_gemm<true, Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >, long> (func=..., rows=100, cols=100, depth=40, transpose=false) at /usr/include/eigen3/Eigen/src/Core/products/Parallelizer.h:97  
#3  0x000000000041a919 in Eigen::internal::generic_product_impl<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::DenseShape, Eigen::DenseShape, 8>::scaleAndAddTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (dst=..., a_lhs=..., a_rhs=..., alpha=@0x7ffff7ff3be0: 1)  
    at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:483  
#4  0x000000000041a2b4 in Eigen::internal::generic_product_impl<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::DenseShape, Eigen::DenseShape, 8>::evalTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (dst=..., lhs=..., rhs=...) at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:435  
#5  0x0000000000419cff in Eigen::internal::Assignment<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::internal::assign_op<double, double>, Eigen::internal::Dense2Dense, void>::run (dst=..., src=...) at /usr/include/eigen3/Eigen/src/Core/ProductEvaluators.h:148  
#6  0x0000000000419753 in Eigen::internal::call_assignment_no_alias<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::internal::assign_op<double, double> > (dst=..., src=..., func=...) at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:836  
#7  0x00000000004193f0 in Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >::_set_noalias<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (this=0x7ffff7ff3d50, other=...) at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:728  
#8  0x0000000000419251 in Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >::_init1<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (this=0x7ffff7ff3d50, other=...)  
    at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:812  
#9  0x000000000041911b in Eigen::Matrix<double, -1, -1, 0, -1, -1>::Matrix<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (  
    this=0x7ffff7ff3d50, x=...) at /usr/include/eigen3/Eigen/src/Core/Matrix.h:296  
#10 0x0000000000418e90 in Eigen::internal::call_assignment<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::internal::assign_op<double, double> > (dst=..., src=..., func=...) at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:796  
#11 0x0000000000418b26 in Eigen::internal::call_assignment<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (dst=..., src=...) at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:782  
#12 0x00000000004187d7 in Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >::_set<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (this=0x62b280 <H>, other=...) at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:710  
#13 0x00000000004184b5 in Eigen::Matrix<double, -1, -1, 0, -1, -1>::operator=<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (  
    this=0x62b280 <H>, other=...) at /usr/include/eigen3/Eigen/src/Core/Matrix.h:225  
#14 0x00000000004171d3 in test_mul (nrows_=100, ncols_=40) at /home/alaurenzi/Code/eigen_xeno3_test/src/main.cpp:103  
#15 0x000000000041723a in regular_thread (arg=0x0) at /home/alaurenzi/Code/eigen_xeno3_test/src/main.cpp:136  
#16 0x00007ffff7bc782c in cobalt_thread_trampoline () from /usr/xenomai/lib/libcobalt.so.2  
#17 0x00007ffff779d6ba in start_thread (arg=0x7ffff7ff4700) at pthread_create.c:333  
#18 0x00007ffff6a2a3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Changing the outer dimension to 20 (global variable nrows in the provided example), I see that [20x176] by [176x20] is ok, while [20x177] by [177x20] fails. Interestingly, the backtrace is different:

Thread 3 "xeno3_tests" received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7ffff7ff4700 (LWP 19481)]  
0x000000000041803c in Eigen::internal::pbroadcast4<double __vector(2)>(Eigen::internal::unpacket_traits<double __vector(2)>::type const*, double __vector(2)&, double __vector(2)&, double __vector(2)&, double __vector(2)&) (a=0x0, a0=..., a1=..., a2=..., a3=...) at /usr/include/eigen3/Eigen/src/Core/arch/SSE/PacketMath.h:477  
477	{  
(gdb) bt  
#0  0x000000000041803c in Eigen::internal::pbroadcast4<double __vector(2)>(Eigen::internal::unpacket_traits<double __vector(2)>::type const*, double __vector(2)&, double __vector(2)&, double __vector(2)&, double __vector(2)&) (a=0x0, a0=..., a1=..., a2=..., a3=...) at /usr/include/eigen3/Eigen/src/Core/arch/SSE/PacketMath.h:477  
#1  0x0000000000421bf5 in Eigen::internal::gebp_traits<double, double, false, false>::broadcastRhs(double const*, double __vector(2)&, double __vector(2)&, double __vector(2)&, double __vector(2)&) (this=0x7ffff7fe510b, b=0x7ffff7fe5990, b0=..., b1=..., b2=..., b3=...) at /usr/include/eigen3/Eigen/src/Core/products/GeneralBlockPanelKernel.h:403  
#2  0x000000000041dddf in Eigen::internal::gebp_kernel<double, double, long, Eigen::internal::blas_data_mapper<double, long, 0, 0>, 4, 4, false, false>::operator() (  
    this=0x7ffff7ff377e, res=..., blockA=0x7ffff7fec850, blockB=0x7ffff7fe5990, rows=20, depth=177, cols=20, alpha=1, strideA=177, strideB=177, offsetA=0, offsetB=0)  
    at /usr/include/eigen3/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1219  
#3  0x000000000041c480 in Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>::run (rows=20, cols=20, depth=177, _lhs=0x7ffff00076d0,   
    lhsStride=20, _rhs=0x7ffff000e580, rhsStride=177, _res=0x7ffff00008c0, resStride=20, alpha=1, blocking=..., info=0x0)  
    at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:194  
#4  0x000000000041b13e in Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >::operator() (this=0x7ffff7ff3b20, row=0, rows=20, col=0, cols=20, info=0x0) at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:226  
#5  0x000000000041ad8c in Eigen::internal::parallelize_gemm<true, Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >, long> (func=..., rows=20, cols=20, depth=177, transpose=false) at /usr/include/eigen3/Eigen/src/Core/products/Parallelizer.h:97  
#6  0x000000000041a919 in Eigen::internal::generic_product_impl<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::DenseShape, Eigen::DenseShape, 8>::scaleAndAddTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (dst=..., a_lhs=..., a_rhs=..., alpha=@0x7ffff7ff3be0: 1)  
    at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:483  
#7  0x000000000041a2b4 in Eigen::internal::generic_product_impl<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::DenseShape, Eigen::DenseShape, 8>::evalTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (dst=..., lhs=..., rhs=...) at /usr/include/eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:435  
#8  0x0000000000419cff in Eigen::internal::Assignment<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::internal::assign_op<double, double>, Eigen::internal::Dense2Dense, void>::run (dst=..., src=...) at /usr/include/eigen3/Eigen/src/Core/ProductEvaluators.h:148  
#9  0x0000000000419753 in Eigen::internal::call_assignment_no_alias<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::internal::assign_op<double, double> > (dst=..., src=..., func=...) at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:836  
#10 0x00000000004193f0 in Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >::_set_noalias<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (this=0x7ffff7ff3d50, other=...) at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:728  
#11 0x0000000000419251 in Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >::_init1<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (this=0x7ffff7ff3d50, other=...)  
    at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:812  
#12 0x000000000041911b in Eigen::Matrix<double, -1, -1, 0, -1, -1>::Matrix<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (  
    this=0x7ffff7ff3d50, x=...) at /usr/include/eigen3/Eigen/src/Core/Matrix.h:296  
#13 0x0000000000418e90 in Eigen::internal::call_assignment<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::internal::assign_op<double, double> > (dst=..., src=..., func=...) at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:796  
#14 0x0000000000418b26 in Eigen::internal::call_assignment<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (dst=..., src=...) at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:782  
#15 0x00000000004187d7 in Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >::_set<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (this=0x62b280 <H>, other=...) at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:710  
#16 0x00000000004184b5 in Eigen::Matrix<double, -1, -1, 0, -1, -1>::operator=<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> > (  
    this=0x62b280 <H>, other=...) at /usr/include/eigen3/Eigen/src/Core/Matrix.h:225  
#17 0x00000000004171d3 in test_mul (nrows_=20, ncols_=177) at /home/alaurenzi/Code/eigen_xeno3_test/src/main.cpp:103  
#18 0x000000000041723a in regular_thread (arg=0x0) at /home/alaurenzi/Code/eigen_xeno3_test/src/main.cpp:136  
#19 0x00007ffff7bc782c in cobalt_thread_trampoline () from /usr/xenomai/lib/libcobalt.so.2  
#20 0x00007ffff779d6ba in start_thread (arg=0x7ffff7ff4700) at pthread_create.c:333  
#21 0x00007ffff6a2a3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

I also tried #define EIGEN_DONT_VECTORIZE before including the headers, nothing changes.

Any clues? Please feel free to ask for further tests if you don't have a Xenomai 3 platform (dual kernel - Cobalt) available.

Attachment 800, "Small self-contained example. Builds with cmake.":
eigen_xeno3_test.zip

Edited Dec 05, 2019 by Eigen Bugzilla