Skip to content

Fix Woverflow warnings in PacketMathFP16

Alexander Grundrequested to merge
Flamefire/eigen:overflowFP16 into master

(At least) when compiling for -march=sapphirerapids I see a large number of warnings:

/git/eigen/Eigen/src/Core/arch/AVX512/PacketMathFP16.h: In function ‘Packet Eigen::internal::pcmp_lt_or_nan(const Packet&, const Packet&) [with Packet = __vector(32) _Float16]’:
/git/eigen/Eigen/src/Core/arch/AVX512/PacketMathFP16.h:293:81: warning: overflow in conversion from ‘unsigned int’ to ‘short int’ changes value from ‘65535’ to ‘-1’ [-Woverflow]
  293 |   return _mm512_castsi512_ph(_mm512_mask_set1_epi16(_mm512_set1_epi16(0), mask, 0xffffu));
      |                                                                                 ^~~~~~~

_mm512_mask_set1_epi16 is at some places documented as taking an int there but the Intel intrinsics guide lists it as short.

This PR adds an explicit cast to short.

Where _mm512_mask_set1_epi16 is taking a short the behavior is unchanged (besides the warning gone).
Where _mm512_mask_set1_epi16 is taking an int this introduces "implementation defined behavior".

Alternative is to use -1 or disable the warning for those 4 functions.

Merge request reports