Product of 3x3 fixed-size matrix and its inverse gives wrong result

Submitted by ipo..@..tbr.cz

Assigned to Nobody

Link to original bugzilla bug (#1442)
Version: 3.2
Operating system: Windows

Description

Created attachment 792
Test for various sizes of matrices. Does not contain main(),

When taking a product of a 3x3 fixed-size matrix (Eigen::Matrix3d) and its inverse, the output will not be an identity matrix. Looking deeper into the issue, the last row of the inverse is not computed correctly. Adding an eval() solves the issue.

typedef Eigen::Matrix3d Matrix;  
Matrix M = Matrix::Random();  
Matrix I_bad = M * M.inverse(); // buggy  
Matrix I_good = M * M.inverse().eval(); // ok  
Matrix bad_inv = M.inverse().eval() * I_bad; // what M.inverse() evaluated to  
Matrix good_inv = M.inverse(); // what it was supposed to evaluate to  

This does not seem to affect dynamic matrices. A more comprehensive test is attached (I've tested up to 100x100 fixed-size matrices, only 3x3 have the issue). I get:

M =
-0.0402539 0.64568 0.717887
-0.299417 0.49321 0.421003
0.791925 -0.651784 0.0270699

M * M.inverse() =
-1.04439 5.07456 1.81471
-1.19893 3.97596 1.06423
-0.0770894 0.19135 1.06843

the M.inverse() was
4.19313 -7.073 -1.19833
4.97643 -8.3002 -2.88525
-5.69557 14.1375 5.0557

real M.inverse() is
4.19313 -7.073 -1.19833
4.97643 -8.3002 -2.88525
-2.84779 7.06874 2.52785

M * M.inverse().eval() =
1 -8.88178e-016 2.22045e-016
2.22045e-016 1 2.22045e-016
-9.4369e-016 8.88178e-016 1

This happens in Visual Studio 2008 Professional with SP1 on Windows, with Eigen 3.2.10 and only in x86 (32-bit) debug build. It is somewhat elusive, as it does not happen with Visual Studio 2008 Express on another machine. It does not seem to be an issue in Linux (tried with g++ 4.4.7, with Eigen 3.2.4, 3.2.10 and 3.3.3).

Attachment 792, "Test for various sizes of matrices. Does not contain main(),":
Eigen_InvBug.cpp