Skip to content

Improve accuracy of erf().

This implements an improved rational approximation and more careful clamping for the error function erf(). Speed is unchanged.

Table of maximum relative errors in ULPs before and after.

Range Before After
Subnormal floats 250,000 630
Normalized floats 32 3

Thanks to my colleague James Lottes for deriving the rational approximant.

Merge request reports

Loading