Core: Preserve bfloat16 packet sign under FTZ and DAZ

Implement psign directly on bfloat16's 16-bit packet lanes for AVX, AVX-512, NEON, and AltiVec. The integer masks avoid floating-point comparisons/conversions that can flush subnormal operands under FTZ/DAZ. Both signed zeros become +0; nonzero finite values and infinities become signed one; NaN sign and payload bits are preserved.

Add an exhaustive packet regression over all 65,536 bfloat16 encodings, also run with verified input flushing (FZ on ARM64, and FTZ plus DAZ on x86), plus a packet-level benchmark with normal, subnormal, and mixed inputs. This is independent of the float/double change in !3175 (merged).

Reference issue

Fixes #3182 (closed).

Validation

  • packetmath.cpp part 15 passes natively on ARM64 and under Rosetta with AVX and AVX2; the flushed run now asserts input flushing, including DAZ on x86. The AVX-512 variant compiles but could not run on this host. The same regression fails against parent 4cba415f.
  • bench_packet_ops was run for the initial kernel revision using Google Benchmark 1.9.5, but not rerun after the backend simplifications in d603621; the earlier timings are not evidence for this revision.
  • AltiVec execution and native AVX-512 execution were unavailable locally. An earlier broad -ffast-math packetmath run failed at the same unrelated log_fun case on both parent and MR branch; it was not rerun for d603621.
  • check_style.py, git diff --check, and an Apple clang-format 21 check of the changed lines pass at d603621. clang-format-17 is unavailable locally.

Update: Thanks to Rasmus Munk Larsen for catching the inactive x86 DAZ guard. The flushed test now enables DAZ, verifies input flushing, and runs outside the subtest macro argument; the packet specializations explain the encoded-lane comparisons.

Update: Thanks to Charles Schlosser for the shorter AVX, AVX-512, NEON, and AltiVec integer-mask sequences. The exhaustive bfloat16 packet test passes on ARM64 and AVX/AVX2 under Rosetta; AVX-512 compiles, and the AltiVec lane formula was checked against all 65,536 encodings but not executed in Eigen locally.

Update (662311f): Rebased onto upstream 851458df, which includes !3222 (merged)'s ARMv7 NEON subnormal integer-power fix. All four bfloat16 commits are unchanged according to git range-diff. The exhaustive bfloat16 packet regression (packetmath part 15, including input flushing) passes on native ARM64 after the rebase. CI has been restarted on the rebased head; the earlier unary_pow failures were also present on upstream master before !3222 (merged).

The restarted fork pipeline 2891126114 runs smoke tests only. The upstream ARMv7 affected-test job still needs a maintainer-triggered parent-project pipeline; its result has not yet been verified on the rebased head.

Update (a938def): Rebased onto 93933c8, resolving the overlapping additions while retaining the upstream float/double sign regressions and pldexp benchmarks. The bfloat16 kernels are unchanged. With Apple clang 21, packetmath part 15 passes on native ARM64 and AVX/AVX2 under Rosetta; parts 1 and 2 pass on AVX under Rosetta. The packet benchmark passes C++17 syntax checks on ARM64 and AVX, and the changed lines pass clang-format 17.0.6, check_style.py, and git diff --check. Fork smoke pipeline 2891869827 is running on this head; the upstream ARMv7 affected-test rerun still needs a maintainer.

Edited by Saicharan Ramineni

Merge request reports

Loading
Loading