Skip to content

prsqrt returns -inf instead of NaN for negative denormal values on some platforms

It appears that the RSQRTPS and VRSQRTPS instructions for SSE and AVX, respectively, flush denorm to zero. This causes us to return -Inf instead of NaN for negative denormals.

Thanks to majnemer@google.com for reporting this.

Edited by Rasmus Munk Larsen