Improve random (avoid undefined behavior).
Submitted by Alexey Frunze
Assigned to Nobody
Link to original bugzilla bug (#1584)
Version: 3.4 (development)
Operating system: Linux
Description
Created attachment 878
patch with fix
This fixes failures in test rand on MIPS32R5.
The failures are a result of undefined behavior in template<typename Scalar> struct random_default_impl<Scalar, false, true>.
With certain values of arguments x and y there's a signed integer overflow in expression ScalarX(y)-ScalarX(x). This showed with the particular compiler version we're using for MIPS32R5.
Also, the template wouldn't handle the whole range of Scalar when it's 64-bit (long long or __int64) on 32-bit CPUs. This should fix it as well.
Patch 878, "patch with fix":
improve_random.patch
Edited by Eigen Bugzilla