Skip to content

Numerical precision of sum operation on C_CONTIGUOUS arrays

Some core computations are using the np.sum operation on large floating point numbers (e.g. sum of squared traces in Pearson's correlation).

According to numpy's documentation on that matter, "improved precision is only used when the summation is along the fast axis in memory." Which means our computation reaches better precision if the arrays are using Fortran order.
Because most of computations starts with a copy of input data into a float array, we could enforce F_CONTIGUOUS arrays when advisable:

  • Add a test ensuring that concerned operations uses F_CONTIGUOUS arrays regardless of the memory representation order of input data
  • Modify core computation to enforce F_CONTIGUOUS when required.