nestbyvalue test: fix uninitialized matrix

• Doing computation with uninitialized (zero-ed ? but thanks Linux) matrix, or worse NaN on other non-linux systems.
• This commit fixes it by initializing to Random().

Note: can we have this cherry-picked into 3.4 too, please ?