Core: Vectorize complex component reductions (!3108) · Merge requests · libeigen / eigen

Complex real() and imag() reductions gain packet access through the existing stride-2 pgather operation. SSE, AVX/AVX2, and AVX-512 gathers now use bounded overlapping loads and shuffles for stride 2, removing the GCC float PropagateFast regression: 4,096 elements take 251 ns versus 766 ns in the previous MR revision and 569 ns before the MR. Small Clang inputs and GCC double PropagateFast still cost more than the original baseline, up to 32% for GCC/SSE2 at 4,096 elements. The double cost comes from the two-accumulator loop in Redux.h, which !3116 (merged) removes independently of this MR; the measurements below include both.

c.real().cwiseAbs().maxCoeff<Policy>(), mutable Matrix<std::complex<T>, Dynamic, 1>, AVX2; CPU ns, median ± MAD, 14 samples per binary:

Compiler	T	Policy	Elements	Original baseline (ns)	Current head (ns)	Result
Gcc	float	PropagateFast	4,096	568.823 ± 7.615	251.110 ± 5.311	2.27× faster
Gcc	float	PropagateNaN	4,096	1932.377 ± 40.438	476.877 ± 4.242	4.05× faster
Gcc	double	PropagateNaN	4,096	1900.498 ± 16.540	941.632 ± 23.022	2.02× faster
Clang	float	PropagateFast	4,096	790.597 ± 24.434	253.550 ± 10.161	3.12× faster
Clang	float	PropagateNaN	4,096	1700.117 ± 36.458	471.227 ± 7.615	3.61× faster
Clang	double	PropagateNaN	4,096	1891.912 ± 22.671	1028.572 ± 64.418	1.84× faster
Gcc	double	PropagateFast	4,096	598.929 ± 10.739	657.217 ± 15.283	9.7% slower
Gcc	double	PropagateFast	65,536	10452.574 ± 151.717	12189.534 ± 602.884	16.6% slower
Clang	double	PropagateFast	65,536	11414.947 ± 444.405	12270.178 ± 363.990	7.5% slower
Clang	float	PropagateNaN	3	1.994 ± 0.039	2.745 ± 0.123	37.7% slower

Original baseline: 20f5a26ba4d90b17762ed78415dad33afb17e9a3; previous MR revision: df51208c471af25a3e3b4c8411eae7594d804ef1; current head: a40b378293e94de417b9ca42d846985697db683d (since rebased to 4c2a822ffd8d08cd7dab29e30ad58121ee8c10d8 with this MR's diff unchanged). Local CPU: 13th Gen Intel(R) Core(TM) i7-13700HX, WSL2, affinity 2; GCC 13.3 / Clang 18.1.3, C++14, -O3 -DNDEBUG -mavx2 -mfma. Binaries ran separately in the order original/previous/current, twice, seven repetitions per invocation, with identical Google Benchmark libraries and no concurrent builds/tests. Frequency and thermals were not locked.

The stride-2 optimization itself, compared with the previous MR revision (df51208c4), at 4,096 elements:

Compiler / ISA / CPU	T	Policy	Elements	Previous MR (ns)	Current head (ns)	Result
GCC / SSE2 / Intel	float	PropagateFast	4,096	673.206 ± 14.682	464.933 ± 33.553	1.45× faster
GCC / AVX / Intel	float	PropagateFast	4,096	862.193 ± 29.704	305.943 ± 12.737	2.82× faster
GCC / AVX2 / Intel	float	PropagateFast	4,096	766.477 ± 8.752	251.110 ± 5.311	3.05× faster
GCC / AVX-512DQ / Zen 5	float	PropagateFast	4,096	1117.452 ± 3.710	80.311 ± 1.216	13.91× faster
GCC / AVX-512DQ / Zen 5	float	PropagateNaN	4,096	1179.720 ± 3.082	145.940 ± 1.503	8.08× faster
GCC / AVX-512DQ / Zen 5	double	PropagateFast	4,096	1748.583 ± 6.668	428.376 ± 10.051	4.08× faster
GCC / AVX-512DQ / Zen 5	double	PropagateNaN	4,096	1806.651 ± 4.422	411.596 ± 6.681	4.39× faster

Zen 5 CPU: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, WSL2, affinity 30, GCC 13.3, -mavx512f -mavx512dq -mfma. Only one short benchmark process ran at a time alongside an existing single-threaded benchmark; all compilation was local. Two previous/current pairs, seven repetitions per invocation, agree on the large gains. These measurements compare revisions on each CPU, not the CPUs with each other.

Double `PropagateFast` and !3116 (merged)

Update: @chuckyschluz traced the remaining double PropagateFast cost to redux_impl<Func, Evaluator, LinearVectorizedTraversal, NoUnrolling> rather than the gather. That loop carries two packet accumulators, so its throughput is capped by maxpd latency, and GCC/SSE2 — absent from the grids above — pays the most. !3116 (merged) uses four accumulators and does not depend on this MR. double / real / mutable / PropagateFast, re-measured at 4c2a822ff:

Compiler / ISA	Elements	Original (ns)	Head (ns)	Head + !3116 (merged) (ns)	Original/head	Original/(head + !3116 (merged))
GCC / SSE2	4,096	653.467 ± 37.990	861.253 ± 10.562	584.826 ± 4.840	0.76×	1.12×
GCC / SSE2	65,536	12338.598 ± 1259.960	14246.835 ± 304.546	10754.815 ± 92.478	0.87×	1.15×
GCC / AVX2	4,096	588.999 ± 4.768	632.204 ± 3.168	594.253 ± 14.509	0.93×	0.99×
GCC / AVX2	65,536	10922.796 ± 683.059	11517.128 ± 128.536	11019.114 ± 233.457	0.95×	0.99×
Clang / AVX2	4,096	654.908 ± 25.421	530.481 ± 4.020	462.278 ± 7.479	1.23×	1.42×
Clang / AVX2	65,536	11011.593 ± 188.140	9501.072 ± 365.922	9072.209 ± 410.660	1.16×	1.21×

Headers 20f5a26ba, 4c2a822ff, and 4c2a822ff merged with !3116 (merged) at b8c5953a3; same CPU, compilers, and benchmark source as above, with the ISA flags listed in the methodology appendix. Two rounds (original/head/merged, then reversed) of seven repetitions give 14 samples per binary; each binary ran in its own invocation once the 1-minute load average fell below 0.85. With !3116 (merged), the cases still below the original baseline across the 60-case grids are GCC/SSE2 double PropagateFast at length 3 (0.95×), the two GCC/AVX2 rows above (0.99×), and the Clang/AVX2 length-3 cases (0.69–0.93×). GCC/AVX2 double PropagateFast at 4,096 is placement-sensitive: 657 ns and 514 ns in the two earlier runs, 632 ns here. Complete grids are in the appendix below.

Reference issue

Closes #3097 (closed). Implements the remaining component-vectorization improvement identified in the report and follow-up measurements by @rmlarsen1.

Description

Gather a component from direct std::complex<float/double> storage with compile-time unit inner stride, using pgather<Scalar, Packet>(data + 2 * index + Component, 2). Other supported packet backends reuse their existing gather implementations; there is no new packet operation.
Select const evaluators for reductions. Writable views retain scalar stores, and strided, computed, and custom-scalar inputs retain the generic evaluator. Blocks honor evaluator direct-access flags so component packet reads delegate to the component evaluator.
Optimize existing stride-2 gathers: SSE float uses two loads and a shuffle; AVX float combines SSE gathers; AVX2 float/double use two loads, a shuffle, and a permutation; AVX-512 float/double use two loads and a two-source permutation. Loads stop at from[2 * (PacketSize - 1)], including imaginary components at the allocation boundary. Other strides and masked gather implementations are unchanged.

GCC's previous float loop spent instructions loading and inserting individual components into packets; the revised loop deinterleaves contiguous loads. AVX-512 replaces hardware gathers in this stride-2 case. Existing packet min/max operations preserve NaN behavior. Callgrind records 56% fewer instructions for the AVX2 float PropagateFast driver and 37% fewer for SSE; its complete counts are below.

The full AVX2 grid shows GCC double PropagateFast costs of 9.7% at 4,096 and 16.6% at 65,536 versus the original baseline, and Clang double costs 7.5% at 65,536; the !3116 (merged) re-measurement above puts GCC at 0.93× and 0.95× and Clang at 1.16× for those cases. Clang length-3 cases cost 12–42% (0.17–0.83 ns). Large-double effects from the new gather optimization vary between runs: a longer targeted repeat improved PropagateFast, while the initial Clang PropagateNaN regression versus the previous MR did not reproduce. Both sets are retained below; large-double speedups are inconclusive.

Validation

112 CTest cases passed at the current implementation, plus focused native tests, four ASan+UBSan runs, six Valgrind Memcheck runs with zero errors, and ten native AVX-512 test executables on Zen 5. Builds used -j4 and a 4 GiB virtual-memory cap per compiler process.

The component tests cover both components and storage orders, const/mutable expressions, blocks/transposes, fixed/dynamic sizes, unaligned maps, strides, writable-view isolation, NaN policies, infinities, signed zeros, subnormals, and custom scalars. Packet-access assertions fail at the original baseline. The new gather tests exercise full and half packets, literal/runtime stride 2, negative/zero/other strides, alignment offsets, exact allocation bounds, and bitwise preservation including NaN payloads. The new benchmark measures both literal and runtime gathers, with strides 1/3/7 as controls.

AVX-512F-only gather tests compile and run. The component benchmark with AVX-512F but without DQ fails in the existing Reductions.h use of _mm512_extractf32x8_ps; the same failure occurs with the previous MR headers. AVX-512F/DQ component and packet tests pass natively. Installed Valgrind 3.22 cannot execute AVX-512 (the probe exits with an unsupported-instruction failure), so Memcheck and Callgrind cover SSE/AVX/AVX2 and Zen 5 supplies AVX-512 execution. No tests or backends were disabled.

CI selection remains affected-tests. The SSE/AVX/AVX512 paths select the relevant additional x86 jobs; no all-platforms or all-tests label is added.

Update: Following @rmlarsen1's review, the evaluator reuses pgather. The follow-up GCC float investigation now optimizes that existing operation across SSE, AVX/AVX2, and AVX-512, with native Zen 5 validation and Valgrind measurements as requested.

Validation matrix and reproduction

All correctness tests use C++14, seed 1, repeat 2, except Memcheck uses repeat 1.

Configuration	Tests	Result
GCC 13.3, AVX2/FMA, Release	`complex_components_1/2/3`, `packetmath_1/2`, `packetmath_gather_1/2`, all parts of `redux`, `realview`, `block`, `vectorization_logic`, `array_cwise`, `ref`, `special_packetmath_1/2`	100/100 CTest
Clang 18.1.3, SSE2, Release	`complex_components_1/2/3`, `packetmath_1/2`, `packetmath_gather_1/2`	7/7 CTest
GCC 13.3, explicit vectorization disabled	`complex_components_1/2/3`, `packetmath_gather_1/2`	5/5 CTest
GCC 13.3, SSE2 / AVX without AVX2 / AVX2	`packetmath_gather_1/2` in each configuration	6 native runs, plus 6 Memcheck runs; zero errors
Clang 18.1.3, AVX without AVX2	`packetmath_gather_1/2`	2 native runs
Clang 18.1.3, SSE2 / AVX2, ASan+UBSan, `-O1`	`packetmath_gather_1/2` in each configuration	4 runs; leak checking disabled
GCC 13.3, AVX-512F without DQ, Zen 5	`packetmath_gather_1/2`	2 native runs
GCC 13.3, AVX-512F/DQ, Zen 5	`packetmath_gather_1/2`, `complex_components_1/2`, `packetmath_1/2`	6 native runs
Clang 18.1.3, AVX-512F/DQ, Zen 5	`packetmath_gather_1/2`	2 native runs

Main CMake configuration and checks:

cmake -G Ninja -S . -B build-avx2 -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_FLAGS='-mavx2 -mfma' -DEIGEN_BUILD_DOC=OFF \
  -DEIGEN_BUILD_BLAS=OFF -DEIGEN_BUILD_LAPACK=OFF
cmake --build build-avx2 -j4 --target complex_components packetmath_1 packetmath_2 \
  packetmath_gather redux realview block vectorization_logic array_cwise ref \
  special_packetmath_1 special_packetmath_2
EIGEN_REPEAT=2 EIGEN_SEED=1 ctest --test-dir build-avx2 -j4 \
  -R '^(complex_components_[123]|packetmath_[12]|packetmath_gather_[12]|redux_[0-9]+|realview_[0-9]+|block_[0-9]+|vectorization_logic_[0-9]+|array_cwise_[0-9]+|ref_[0-9]+|special_packetmath_[12])$' \
  --output-on-failure --no-tests=error
valgrind --tool=memcheck --error-exitcode=99 --leak-check=full ./gather-test r1 s1
ASAN_OPTIONS=detect_leaks=0 ./gather-asan-test r2 s1

Clang uses a separate Release directory with -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang; the scalar configuration uses -DEIGEN_TEST_NO_EXPLICIT_VECTORIZATION=ON. Standalone tests compile test/packetmath_gather.cpp through its existing umbrella-based harness, with -std=c++14 -O2 -I. -DEIGEN_TEST_PART_1 or _2 and the relevant ISA flags. Sanitizers use -O1 -fsanitize=address,undefined -fno-omit-frame-pointer.

Changed-line clang-format-17, new-file formatting, style, spelling, REUSE metadata, and git diff --check passed. Clang-tidy found no issues on changed header lines; the AVX/AVX512 drivers were checked with AVX-512F/DQ enabled because the baseline driver cannot parse those private headers without the ISA flags.

At previous revision df51208c4, the unchanged generic component evaluator also passed float/double component tests under QEMU for ARM32 NEON, AArch64 NEON and SVE 256, PowerPC POWER9 AltiVec/VSX, RISC-V RVV 256, and LoongArch LSX, plus Clang generic 32-byte packets (14 tests). Those non-x86 configurations were not rerun for this x86 gather change. ZVector remains blocked by an include-only parent failure in integer packet traits used by frexp_generic_get_biased_exponent. Native non-x86 timing, GPU, SYCL, HVX, MSA, and MSVC coverage remain unavailable.

Benchmark methodology and reproduction

bench_complex_components measures c.real()/imag().cwiseAbs().maxCoeff<Policy>() with float/double, const/mutable inputs, and sizes 3, 8, 9, 64, 4096, and 65536. PropagateFast controls use mutable real components. bench_pgather gathers an input into contiguous output, covering literal stride 2 and runtime strides 1/2/3/7 at 64 and 4096 coefficients. Input generation and scalar-result checks are outside timing.

cmake -G Ninja -S benchmarks -B build-bench -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_FLAGS='-mavx2 -mfma -std=c++14'
cmake --build build-bench --target bench_complex_components bench_pgather -j4
# Compile the current benchmark source against each header revision identically:
c++ -O3 -DNDEBUG -DBENCHMARK_STATIC_DEFINE -std=c++14 -mavx2 -mfma \
  -I /path/to/header-tree benchmarks/Core/bench_complex_components.cpp \
  /usr/local/lib/libbenchmark.a /usr/local/lib/libbenchmark_main.a \
  /usr/local/lib/libbenchmark.a -lrt -lpthread -o bench
taskset -c 2 ./bench --benchmark_min_time=0.005s --benchmark_repetitions=7 \
  --benchmark_display_aggregates_only=true --benchmark_out=results.json --benchmark_out_format=json

Use bench_pgather.cpp for the standalone gather benchmark. ISA flags are -msse2 -mno-avx (SSE2), -mavx -mno-avx2 (AVX), -mavx2 -mfma (AVX2), or -mavx512f -mavx512dq -mfma (AVX-512). The full AVX2 grid ran original/previous/current twice; other comparisons ran previous/current twice. Local runs used 0.005 s minimum and 7 repetitions per invocation. The longer double repeat used 0.025 s and 9 repetitions (18 samples). Zen 5 used 0.01 s and 7 repetitions for components (14 samples), 0.005 s and 5 repetitions for gathers (10 samples).

Tables report median ± median absolute deviation (MAD) of CPU ns across individual repetitions; aggregate rows are not counted as samples. Original means 20f5a26ba, Previous means df51208c4, and Current means the head above. Ratios below 1 are regressions. Raw JSON retains individual samples and Google Benchmark aggregates; no significance test was performed. Host load and uncontrolled frequency limit interpretation of small differences. Control strides are retained even where unchanged code shows noisy differences.

gcc-avx2-components: complete measurements (14 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Original (ns)	Previous (ns)	Current (ns)	Original/current	Previous/current
float	real	mutable	PropagateFast	3	1.065 ± 0.020	0.920 ± 0.015	0.874 ± 0.020	1.22×	1.05×
float	real	mutable	PropagateFast	8	1.948 ± 0.051	1.937 ± 0.033	1.404 ± 0.027	1.39×	1.38×
float	real	mutable	PropagateFast	9	2.253 ± 0.080	2.116 ± 0.060	1.805 ± 0.027	1.25×	1.17×
float	real	mutable	PropagateFast	64	9.287 ± 0.311	12.139 ± 0.209	3.664 ± 0.143	2.53×	3.31×
float	real	mutable	PropagateFast	4096	568.823 ± 7.615	766.477 ± 8.752	251.110 ± 5.311	2.27×	3.05×
float	real	mutable	PropagateFast	65536	9178.299 ± 142.346	11816.838 ± 108.125	5282.511 ± 150.213	1.74×	2.24×
float	real	mutable	PropagateNaN	3	1.825 ± 0.026	1.761 ± 0.010	1.814 ± 0.021	1.01×	0.97×
float	real	mutable	PropagateNaN	8	4.600 ± 0.096	2.777 ± 0.083	2.206 ± 0.036	2.09×	1.26×
float	real	mutable	PropagateNaN	9	5.435 ± 0.337	3.267 ± 0.095	2.883 ± 0.079	1.89×	1.13×
float	real	mutable	PropagateNaN	64	29.938 ± 0.769	12.884 ± 0.301	7.615 ± 0.168	3.93×	1.69×
float	real	mutable	PropagateNaN	4096	1932.377 ± 40.438	763.215 ± 15.723	476.877 ± 4.242	4.05×	1.60×
float	real	mutable	PropagateNaN	65536	30884.124 ± 510.916	11957.608 ± 219.219	7569.890 ± 118.682	4.08×	1.58×
float	real	const	PropagateNaN	3	1.876 ± 0.049	1.828 ± 0.042	1.854 ± 0.028	1.01×	0.99×
float	real	const	PropagateNaN	8	4.529 ± 0.083	2.811 ± 0.051	2.288 ± 0.022	1.98×	1.23×
float	real	const	PropagateNaN	9	5.402 ± 0.239	3.341 ± 0.092	2.741 ± 0.045	1.97×	1.22×
float	real	const	PropagateNaN	64	30.889 ± 0.428	13.001 ± 0.566	7.316 ± 0.102	4.22×	1.78×
float	real	const	PropagateNaN	4096	2006.678 ± 36.949	757.474 ± 12.768	473.079 ± 11.320	4.24×	1.60×
float	real	const	PropagateNaN	65536	31508.236 ± 227.210	12148.294 ± 146.617	7684.037 ± 109.828	4.10×	1.58×
float	imag	mutable	PropagateNaN	3	1.961 ± 0.087	1.854 ± 0.041	1.844 ± 0.050	1.06×	1.01×
float	imag	mutable	PropagateNaN	8	5.577 ± 0.127	2.772 ± 0.071	2.245 ± 0.034	2.48×	1.23×
float	imag	mutable	PropagateNaN	9	6.318 ± 0.139	3.320 ± 0.101	2.839 ± 0.031	2.23×	1.17×
float	imag	mutable	PropagateNaN	64	29.062 ± 0.090	13.018 ± 0.276	6.960 ± 0.095	4.18×	1.87×
float	imag	mutable	PropagateNaN	4096	1949.156 ± 37.200	755.322 ± 23.118	462.967 ± 6.126	4.21×	1.63×
float	imag	mutable	PropagateNaN	65536	30427.216 ± 239.985	12201.971 ± 264.971	7408.680 ± 92.524	4.11×	1.65×
float	imag	const	PropagateNaN	3	1.866 ± 0.062	1.845 ± 0.020	1.806 ± 0.032	1.03×	1.02×
float	imag	const	PropagateNaN	8	4.762 ± 0.237	2.674 ± 0.050	2.164 ± 0.047	2.20×	1.24×
float	imag	const	PropagateNaN	9	5.132 ± 0.094	3.311 ± 0.058	2.872 ± 0.056	1.79×	1.15×
float	imag	const	PropagateNaN	64	29.732 ± 0.282	12.894 ± 0.256	7.375 ± 0.210	4.03×	1.75×
float	imag	const	PropagateNaN	4096	1899.839 ± 11.087	756.194 ± 15.910	461.204 ± 7.315	4.12×	1.64×
float	imag	const	PropagateNaN	65536	30955.279 ± 480.878	11936.078 ± 259.249	7623.562 ± 171.477	4.06×	1.57×
double	real	mutable	PropagateFast	3	1.377 ± 0.114	0.882 ± 0.034	0.913 ± 0.025	1.51×	0.97×
double	real	mutable	PropagateFast	8	2.364 ± 0.210	1.989 ± 0.057	1.964 ± 0.046	1.20×	1.01×
double	real	mutable	PropagateFast	9	2.604 ± 0.136	2.361 ± 0.071	2.523 ± 0.058	1.03×	0.94×
double	real	mutable	PropagateFast	64	8.983 ± 0.148	10.582 ± 0.381	6.179 ± 0.312	1.45×	1.71×
double	real	mutable	PropagateFast	4096	598.929 ± 10.739	667.159 ± 12.241	657.217 ± 15.283	0.91×	1.02×
double	real	mutable	PropagateFast	65536	10452.574 ± 151.717	11095.145 ± 308.605	12189.534 ± 602.884	0.86×	0.91×
double	real	mutable	PropagateNaN	3	2.155 ± 0.017	1.763 ± 0.059	1.741 ± 0.034	1.24×	1.01×
double	real	mutable	PropagateNaN	8	5.977 ± 0.149	2.756 ± 0.061	2.707 ± 0.092	2.21×	1.02×
double	real	mutable	PropagateNaN	9	6.787 ± 0.086	3.427 ± 0.128	3.543 ± 0.052	1.92×	0.97×
double	real	mutable	PropagateNaN	64	29.749 ± 0.306	14.248 ± 0.430	12.043 ± 0.224	2.47×	1.18×
double	real	mutable	PropagateNaN	4096	1900.498 ± 16.540	983.292 ± 24.655	941.632 ± 23.022	2.02×	1.04×
double	real	mutable	PropagateNaN	65536	31681.696 ± 403.472	16767.537 ± 439.284	15712.564 ± 732.008	2.02×	1.07×
double	real	const	PropagateNaN	3	2.215 ± 0.056	1.681 ± 0.016	1.690 ± 0.023	1.31×	0.99×
double	real	const	PropagateNaN	8	5.867 ± 0.076	2.749 ± 0.045	2.707 ± 0.112	2.17×	1.02×
double	real	const	PropagateNaN	9	6.706 ± 0.140	3.762 ± 0.143	3.544 ± 0.058	1.89×	1.06×
double	real	const	PropagateNaN	64	30.141 ± 0.755	13.796 ± 0.278	11.955 ± 0.166	2.52×	1.15×
double	real	const	PropagateNaN	4096	1924.313 ± 22.834	998.589 ± 23.010	935.372 ± 23.851	2.06×	1.07×
double	real	const	PropagateNaN	65536	30637.707 ± 326.453	15961.356 ± 259.045	15783.575 ± 393.776	1.94×	1.01×
double	imag	mutable	PropagateNaN	3	2.221 ± 0.069	1.488 ± 0.018	1.757 ± 0.038	1.26×	0.85×
double	imag	mutable	PropagateNaN	8	6.736 ± 0.216	2.878 ± 0.111	2.852 ± 0.058	2.36×	1.01×
double	imag	mutable	PropagateNaN	9	7.686 ± 0.165	3.480 ± 0.151	3.578 ± 0.079	2.15×	0.97×
double	imag	mutable	PropagateNaN	64	29.076 ± 0.153	14.301 ± 0.517	12.646 ± 0.414	2.30×	1.13×
double	imag	mutable	PropagateNaN	4096	1908.329 ± 12.302	993.648 ± 15.209	940.961 ± 7.820	2.03×	1.06×
double	imag	mutable	PropagateNaN	65536	30623.948 ± 459.174	16255.288 ± 614.423	15402.286 ± 300.549	1.99×	1.06×
double	imag	const	PropagateNaN	3	2.199 ± 0.060	1.524 ± 0.045	1.701 ± 0.037	1.29×	0.90×
double	imag	const	PropagateNaN	8	6.000 ± 0.166	2.736 ± 0.081	2.791 ± 0.068	2.15×	0.98×
double	imag	const	PropagateNaN	9	6.737 ± 0.169	3.697 ± 0.066	3.719 ± 0.126	1.81×	0.99×
double	imag	const	PropagateNaN	64	29.523 ± 0.487	14.223 ± 0.354	12.420 ± 0.264	2.38×	1.15×
double	imag	const	PropagateNaN	4096	1911.532 ± 27.078	962.528 ± 16.374	933.301 ± 13.498	2.05×	1.03×
double	imag	const	PropagateNaN	65536	32640.939 ± 994.152	15799.992 ± 575.262	15776.739 ± 637.329	2.07×	1.00×

clang-avx2-components: complete measurements (14 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Original (ns)	Previous (ns)	Current (ns)	Original/current	Previous/current
float	real	mutable	PropagateFast	3	1.379 ± 0.037	2.170 ± 0.037	1.689 ± 0.055	0.82×	1.28×
float	real	mutable	PropagateFast	8	2.291 ± 0.159	2.458 ± 0.099	1.470 ± 0.024	1.56×	1.67×
float	real	mutable	PropagateFast	9	2.349 ± 0.071	2.659 ± 0.039	1.748 ± 0.044	1.34×	1.52×
float	real	mutable	PropagateFast	64	13.046 ± 0.297	12.226 ± 0.219	3.598 ± 0.150	3.63×	3.40×
float	real	mutable	PropagateFast	4096	790.597 ± 24.434	767.910 ± 26.896	253.550 ± 10.161	3.12×	3.03×
float	real	mutable	PropagateFast	65536	12302.082 ± 240.655	12249.953 ± 359.715	5141.122 ± 120.692	2.39×	2.38×
float	real	mutable	PropagateNaN	3	1.994 ± 0.039	2.865 ± 0.067	2.745 ± 0.123	0.73×	1.04×
float	real	mutable	PropagateNaN	8	5.831 ± 0.251	2.759 ± 0.050	2.360 ± 0.065	2.47×	1.17×
float	real	mutable	PropagateNaN	9	7.525 ± 0.171	3.873 ± 0.084	3.419 ± 0.109	2.20×	1.13×
float	real	mutable	PropagateNaN	64	25.741 ± 0.600	12.822 ± 0.270	7.628 ± 0.087	3.37×	1.68×
float	real	mutable	PropagateNaN	4096	1700.117 ± 36.458	741.097 ± 10.892	471.227 ± 7.615	3.61×	1.57×
float	real	mutable	PropagateNaN	65536	26984.532 ± 466.537	12146.311 ± 274.312	7499.959 ± 162.348	3.60×	1.62×
float	real	const	PropagateNaN	3	2.168 ± 0.115	2.587 ± 0.041	2.649 ± 0.076	0.82×	0.98×
float	real	const	PropagateNaN	8	6.105 ± 0.428	2.930 ± 0.095	2.428 ± 0.088	2.51×	1.21×
float	real	const	PropagateNaN	9	7.641 ± 0.543	3.749 ± 0.062	3.522 ± 0.159	2.17×	1.06×
float	real	const	PropagateNaN	64	26.645 ± 0.481	12.672 ± 0.279	7.420 ± 0.291	3.59×	1.71×
float	real	const	PropagateNaN	4096	1703.901 ± 73.065	742.942 ± 7.852	482.836 ± 6.414	3.53×	1.54×
float	real	const	PropagateNaN	65536	28244.630 ± 1488.118	11949.995 ± 140.141	7715.297 ± 93.069	3.66×	1.55×
float	imag	mutable	PropagateNaN	3	2.054 ± 0.052	2.740 ± 0.055	2.856 ± 0.086	0.72×	0.96×
float	imag	mutable	PropagateNaN	8	5.844 ± 0.175	2.710 ± 0.054	2.513 ± 0.063	2.32×	1.08×
float	imag	mutable	PropagateNaN	9	6.724 ± 0.070	3.835 ± 0.134	3.479 ± 0.054	1.93×	1.10×
float	imag	mutable	PropagateNaN	64	26.423 ± 1.402	12.514 ± 0.223	6.998 ± 0.167	3.78×	1.79×
float	imag	mutable	PropagateNaN	4096	1730.806 ± 39.413	734.358 ± 6.318	464.964 ± 10.956	3.72×	1.58×
float	imag	mutable	PropagateNaN	65536	28300.437 ± 1272.268	12172.706 ± 253.759	7897.196 ± 253.170	3.58×	1.54×
float	imag	const	PropagateNaN	3	2.075 ± 0.049	2.618 ± 0.068	2.759 ± 0.074	0.75×	0.95×
float	imag	const	PropagateNaN	8	5.857 ± 0.307	2.738 ± 0.048	2.310 ± 0.078	2.54×	1.19×
float	imag	const	PropagateNaN	9	7.329 ± 0.166	3.811 ± 0.095	3.543 ± 0.067	2.07×	1.08×
float	imag	const	PropagateNaN	64	26.369 ± 0.682	13.022 ± 0.229	7.777 ± 0.111	3.39×	1.67×
float	imag	const	PropagateNaN	4096	1737.577 ± 58.165	755.213 ± 10.748	482.025 ± 7.411	3.60×	1.57×
float	imag	const	PropagateNaN	65536	27753.259 ± 762.121	12234.863 ± 388.142	7697.321 ± 147.798	3.61×	1.59×
double	real	mutable	PropagateFast	3	1.433 ± 0.087	1.688 ± 0.046	1.603 ± 0.017	0.89×	1.05×
double	real	mutable	PropagateFast	8	2.592 ± 0.105	1.709 ± 0.036	1.786 ± 0.034	1.45×	0.96×
double	real	mutable	PropagateFast	9	2.746 ± 0.096	2.184 ± 0.037	2.324 ± 0.080	1.18×	0.94×
double	real	mutable	PropagateFast	64	11.549 ± 0.352	10.360 ± 0.379	8.520 ± 0.311	1.36×	1.22×
double	real	mutable	PropagateFast	4096	676.028 ± 26.273	652.490 ± 15.789	685.170 ± 20.598	0.99×	0.95×
double	real	mutable	PropagateFast	65536	11414.947 ± 444.405	10991.367 ± 272.009	12270.178 ± 363.990	0.93×	0.90×
double	real	mutable	PropagateNaN	3	2.048 ± 0.056	2.630 ± 0.111	2.682 ± 0.070	0.76×	0.98×
double	real	mutable	PropagateNaN	8	5.156 ± 0.439	2.912 ± 0.078	3.049 ± 0.051	1.69×	0.95×
double	real	mutable	PropagateNaN	9	5.906 ± 0.192	3.943 ± 0.150	4.224 ± 0.348	1.40×	0.93×
double	real	mutable	PropagateNaN	64	31.443 ± 0.764	13.971 ± 0.657	14.890 ± 0.668	2.11×	0.94×
double	real	mutable	PropagateNaN	4096	1891.912 ± 22.671	944.674 ± 8.904	1028.572 ± 64.418	1.84×	0.92×
double	real	mutable	PropagateNaN	65536	32014.334 ± 697.385	15493.806 ± 147.904	17800.497 ± 1203.775	1.80×	0.87×
double	real	const	PropagateNaN	3	2.044 ± 0.061	2.667 ± 0.087	2.661 ± 0.038	0.77×	1.00×
double	real	const	PropagateNaN	8	5.128 ± 0.200	2.821 ± 0.124	3.001 ± 0.057	1.71×	0.94×
double	real	const	PropagateNaN	9	6.044 ± 0.167	3.762 ± 0.093	4.068 ± 0.199	1.49×	0.92×
double	real	const	PropagateNaN	64	33.040 ± 0.781	14.218 ± 0.669	14.840 ± 0.279	2.23×	0.96×
double	real	const	PropagateNaN	4096	1897.802 ± 54.691	996.084 ± 32.213	1050.415 ± 61.672	1.81×	0.95×
double	real	const	PropagateNaN	65536	31378.634 ± 1134.581	15925.174 ± 398.251	17522.864 ± 1206.775	1.79×	0.91×
double	imag	mutable	PropagateNaN	3	1.986 ± 0.059	2.621 ± 0.091	2.816 ± 0.119	0.71×	0.93×
double	imag	mutable	PropagateNaN	8	4.942 ± 0.054	2.899 ± 0.108	3.096 ± 0.179	1.60×	0.94×
double	imag	mutable	PropagateNaN	9	6.251 ± 0.219	4.101 ± 0.186	4.049 ± 0.051	1.54×	1.01×
double	imag	mutable	PropagateNaN	64	32.211 ± 0.733	14.267 ± 0.493	13.942 ± 0.382	2.31×	1.02×
double	imag	mutable	PropagateNaN	4096	1916.523 ± 62.276	966.770 ± 22.724	991.144 ± 36.908	1.93×	0.98×
double	imag	mutable	PropagateNaN	65536	29628.784 ± 1212.262	16467.851 ± 390.571	17306.646 ± 1634.673	1.71×	0.95×
double	imag	const	PropagateNaN	3	2.100 ± 0.070	2.727 ± 0.105	2.682 ± 0.031	0.78×	1.02×
double	imag	const	PropagateNaN	8	5.133 ± 0.078	2.764 ± 0.059	3.087 ± 0.110	1.66×	0.90×
double	imag	const	PropagateNaN	9	5.869 ± 0.226	4.228 ± 0.252	4.214 ± 0.133	1.39×	1.00×
double	imag	const	PropagateNaN	64	32.244 ± 1.395	14.724 ± 0.426	15.108 ± 0.599	2.13×	0.97×
double	imag	const	PropagateNaN	4096	1888.142 ± 85.230	983.775 ± 11.750	1024.753 ± 13.804	1.84×	0.96×
double	imag	const	PropagateNaN	65536	30439.745 ± 1476.737	15746.349 ± 388.031	17969.436 ± 892.769	1.69×	0.88×

gcc-sse-components: complete measurements (14 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Previous (ns)	Current (ns)	Previous/current
float	real	mutable	PropagateFast	64	11.065 ± 0.279	5.744 ± 0.134	1.93×
float	real	mutable	PropagateFast	4096	673.206 ± 14.682	464.933 ± 33.553	1.45×
float	real	mutable	PropagateFast	65536	10519.910 ± 205.528	7540.094 ± 560.290	1.40×
float	real	mutable	PropagateNaN	64	13.686 ± 0.494	10.999 ± 0.400	1.24×
float	real	mutable	PropagateNaN	4096	904.562 ± 33.405	842.153 ± 20.508	1.07×
float	real	mutable	PropagateNaN	65536	14336.250 ± 582.089	14034.879 ± 546.233	1.02×
double	real	mutable	PropagateFast	64	10.054 ± 0.404	9.750 ± 0.627	1.03×
double	real	mutable	PropagateFast	4096	914.295 ± 25.584	926.208 ± 40.392	0.99×
double	real	mutable	PropagateFast	65536	15054.280 ± 826.961	15447.958 ± 1357.639	0.97×
double	real	mutable	PropagateNaN	64	19.650 ± 0.488	20.065 ± 0.903	0.98×
double	real	mutable	PropagateNaN	4096	1727.680 ± 70.149	1751.007 ± 106.220	0.99×
double	real	mutable	PropagateNaN	65536	28095.400 ± 1557.762	27671.755 ± 1462.988	1.02×

gcc-avx-components: complete measurements (14 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Previous (ns)	Current (ns)	Previous/current
float	real	mutable	PropagateFast	64	14.214 ± 0.454	5.145 ± 0.251	2.76×
float	real	mutable	PropagateFast	4096	862.193 ± 29.704	305.943 ± 12.737	2.82×
float	real	mutable	PropagateFast	65536	13490.239 ± 278.242	5579.012 ± 144.639	2.42×
float	real	mutable	PropagateNaN	64	14.588 ± 0.462	9.649 ± 0.403	1.51×
float	real	mutable	PropagateNaN	4096	858.468 ± 44.365	530.902 ± 15.752	1.62×
float	real	mutable	PropagateNaN	65536	14229.282 ± 503.995	8759.224 ± 186.922	1.62×
double	real	mutable	PropagateFast	64	12.351 ± 0.407	12.732 ± 0.155	0.97×
double	real	mutable	PropagateFast	4096	739.216 ± 51.208	712.807 ± 26.977	1.04×
double	real	mutable	PropagateFast	65536	12607.017 ± 540.121	12898.495 ± 313.879	0.98×
double	real	mutable	PropagateNaN	64	15.583 ± 0.709	17.677 ± 0.289	0.88×
double	real	mutable	PropagateNaN	4096	1137.523 ± 46.466	1134.710 ± 54.134	1.00×
double	real	mutable	PropagateNaN	65536	18616.687 ± 432.604	18597.711 ± 445.547	1.00×

gcc-sse-pgather: complete measurements (14 samples per binary, ns ± MAD)

T	Stride kind	Stride	N	Previous (ns)	Current (ns)	Previous/current
float	literal	2	64	11.700 ± 0.260	4.445 ± 0.247	2.63×
float	literal	2	4096	758.914 ± 28.214	250.562 ± 10.013	3.03×
double	literal	2	64	9.370 ± 0.090	9.042 ± 0.344	1.04×
double	literal	2	4096	718.455 ± 24.771	749.411 ± 33.777	0.96×
float	runtime	1	64	11.203 ± 0.456	11.204 ± 0.384	1.00×
float	runtime	1	4096	718.323 ± 29.376	746.045 ± 68.204	0.96×
float	runtime	2	64	11.893 ± 0.114	5.286 ± 0.151	2.25×
float	runtime	2	4096	772.647 ± 8.726	300.813 ± 10.799	2.57×
float	runtime	3	64	11.373 ± 0.560	11.072 ± 0.544	1.03×
float	runtime	3	4096	727.144 ± 16.130	748.363 ± 67.026	0.97×
float	runtime	7	64	11.939 ± 0.894	11.629 ± 0.568	1.03×
float	runtime	7	4096	766.541 ± 47.797	804.421 ± 40.077	0.95×
double	runtime	1	64	13.835 ± 1.049	9.304 ± 0.314	1.49×
double	runtime	1	4096	696.213 ± 23.621	668.692 ± 6.365	1.04×
double	runtime	2	64	9.345 ± 0.324	9.280 ± 0.216	1.01×
double	runtime	2	4096	703.056 ± 11.164	738.277 ± 17.059	0.95×
double	runtime	3	64	9.151 ± 0.654	9.357 ± 0.325	0.98×
double	runtime	3	4096	779.157 ± 44.975	786.578 ± 63.839	0.99×
double	runtime	7	64	9.305 ± 0.388	9.115 ± 0.239	1.02×
double	runtime	7	4096	1345.439 ± 55.960	1383.465 ± 71.233	0.97×

gcc-avx-pgather: complete measurements (14 samples per binary, ns ± MAD)

T	Stride kind	Stride	N	Previous (ns)	Current (ns)	Previous/current
float	literal	2	64	12.018 ± 0.326	3.860 ± 0.048	3.11×
float	literal	2	4096	760.853 ± 31.937	226.729 ± 9.543	3.36×
double	literal	2	64	10.224 ± 0.191	10.759 ± 0.268	0.95×
double	literal	2	4096	691.216 ± 18.195	690.812 ± 8.245	1.00×
float	runtime	1	64	12.322 ± 0.131	12.408 ± 0.184	0.99×
float	runtime	1	4096	1027.902 ± 247.124	784.254 ± 16.707	1.31×
float	runtime	2	64	11.778 ± 0.403	4.340 ± 0.203	2.71×
float	runtime	2	4096	761.444 ± 23.578	252.243 ± 7.342	3.02×
float	runtime	3	64	12.712 ± 0.360	12.412 ± 0.183	1.02×
float	runtime	3	4096	776.403 ± 18.375	787.456 ± 8.637	0.99×
float	runtime	7	64	11.574 ± 0.235	11.842 ± 0.314	0.98×
float	runtime	7	4096	777.228 ± 15.610	767.355 ± 24.262	1.01×
double	runtime	1	64	10.341 ± 0.218	10.345 ± 0.091	1.00×
double	runtime	1	4096	691.803 ± 37.539	646.887 ± 11.217	1.07×
double	runtime	2	64	10.417 ± 0.355	11.432 ± 0.495	0.91×
double	runtime	2	4096	683.156 ± 17.559	711.460 ± 6.881	0.96×
double	runtime	3	64	10.753 ± 0.367	11.028 ± 0.543	0.98×
double	runtime	3	4096	761.018 ± 21.713	736.211 ± 13.607	1.03×
double	runtime	7	64	10.839 ± 0.270	10.670 ± 0.413	1.02×
double	runtime	7	4096	1164.341 ± 13.149	1168.502 ± 6.596	1.00×

gcc-avx2-pgather: complete measurements (14 samples per binary, ns ± MAD)

T	Stride kind	Stride	N	Previous (ns)	Current (ns)	Previous/current
float	literal	2	64	11.812 ± 0.317	3.044 ± 0.058	3.88×
float	literal	2	4096	771.951 ± 18.671	190.445 ± 5.317	4.05×
double	literal	2	64	10.262 ± 0.162	8.643 ± 0.117	1.19×
double	literal	2	4096	696.029 ± 23.291	660.423 ± 19.662	1.05×
float	runtime	1	64	12.374 ± 0.146	12.367 ± 0.358	1.00×
float	runtime	1	4096	781.400 ± 7.300	784.708 ± 9.458	1.00×
float	runtime	2	64	11.862 ± 0.351	3.585 ± 0.084	3.31×
float	runtime	2	4096	746.701 ± 9.735	200.285 ± 9.079	3.73×
float	runtime	3	64	11.714 ± 0.215	11.782 ± 0.243	0.99×
float	runtime	3	4096	751.348 ± 13.162	765.390 ± 15.879	0.98×
float	runtime	7	64	12.198 ± 0.335	12.192 ± 0.515	1.00×
float	runtime	7	4096	763.571 ± 12.119	760.247 ± 20.246	1.00×
double	runtime	1	64	10.168 ± 0.096	10.254 ± 0.119	0.99×
double	runtime	1	4096	646.049 ± 10.534	662.296 ± 10.078	0.98×
double	runtime	2	64	10.592 ± 0.328	8.336 ± 0.328	1.27×
double	runtime	2	4096	697.022 ± 25.892	645.125 ± 4.510	1.08×
double	runtime	3	64	10.380 ± 0.270	10.413 ± 0.138	1.00×
double	runtime	3	4096	709.249 ± 13.866	732.927 ± 12.704	0.97×
double	runtime	7	64	10.759 ± 0.448	10.543 ± 0.537	1.02×
double	runtime	7	4096	1235.730 ± 31.371	1160.224 ± 23.984	1.07×

zen5-avx512-components: complete measurements (14 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Previous (ns)	Current (ns)	Previous/current
float	real	mutable	PropagateFast	4096	1117.452 ± 3.710	80.311 ± 1.216	13.91×
float	real	mutable	PropagateNaN	4096	1179.720 ± 3.082	145.940 ± 1.503	8.08×
double	real	mutable	PropagateFast	4096	1748.583 ± 6.668	428.376 ± 10.051	4.08×
double	real	mutable	PropagateNaN	4096	1806.651 ± 4.422	411.596 ± 6.681	4.39×

zen5-avx512-pgather: complete measurements (10 samples per binary, ns ± MAD)

T	Stride kind	Stride	N	Previous (ns)	Current (ns)	Previous/current
float	literal	2	4096	1010.467 ± 2.784	105.374 ± 1.963	9.59×
double	literal	2	4096	1513.158 ± 8.081	556.793 ± 19.133	2.72×
float	runtime	1	4096	1010.122 ± 3.106	1026.074 ± 19.479	0.98×
float	runtime	2	4096	1012.489 ± 3.067	102.154 ± 0.775	9.91×
float	runtime	3	4096	1045.601 ± 28.785	1011.922 ± 4.316	1.03×
float	runtime	7	4096	1017.035 ± 4.480	1014.488 ± 8.527	1.00×
double	runtime	1	4096	1514.639 ± 8.125	1506.811 ± 3.653	1.01×
double	runtime	2	4096	1505.893 ± 2.996	562.318 ± 14.182	2.68×
double	runtime	3	4096	1510.791 ± 9.537	1513.294 ± 13.392	1.00×
double	runtime	7	4096	1513.577 ± 6.107	1506.076 ± 7.604	1.00×

gcc-double-confirmation: complete measurements (18 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Previous (ns)	Current (ns)	Previous/current
double	real	mutable	PropagateFast	64	10.546 ± 0.094	6.837 ± 0.320	1.54×
double	real	mutable	PropagateFast	4096	655.549 ± 10.036	514.331 ± 4.706	1.27×
double	real	mutable	PropagateFast	65536	11140.001 ± 334.086	9402.102 ± 125.986	1.18×
double	real	mutable	PropagateNaN	64	13.925 ± 0.217	12.145 ± 0.205	1.15×
double	real	mutable	PropagateNaN	4096	966.291 ± 11.304	916.211 ± 10.981	1.05×
double	real	mutable	PropagateNaN	65536	15490.180 ± 170.386	15226.197 ± 195.787	1.02×

clang-double-confirmation: complete measurements (18 samples per binary, ns ± MAD)

T	Component	Input	Policy	N	Previous (ns)	Current (ns)	Previous/current
double	real	mutable	PropagateFast	64	10.319 ± 0.071	8.027 ± 0.205	1.29×
double	real	mutable	PropagateFast	4096	643.006 ± 6.850	559.474 ± 5.488	1.15×
double	real	mutable	PropagateFast	65536	11179.946 ± 206.298	9632.258 ± 317.981	1.16×
double	real	mutable	PropagateNaN	64	14.259 ± 0.404	14.159 ± 0.187	1.01×
double	real	mutable	PropagateNaN	4096	979.640 ± 14.636	977.279 ± 8.425	1.00×
double	real	mutable	PropagateNaN	65536	16016.100 ± 376.724	16037.186 ± 518.001	1.00×

!3116 (merged) re-measurement: complete grids (14 samples per binary, ns ± MAD)

GCC / SSE2

T	Component	Input	Policy	N	Original (ns)	Head (ns)	Head + !3116 (merged) (ns)	Original/head	Original/(head + !3116 (merged))
float	real	mutable	PropagateFast	3	1.211 ± 0.024	0.832 ± 0.011	0.864 ± 0.007	1.46×	1.40×
float	real	mutable	PropagateFast	8	3.940 ± 1.688	1.652 ± 0.020	1.346 ± 0.012	2.38×	2.93×
float	real	mutable	PropagateFast	9	2.612 ± 0.138	2.069 ± 0.023	1.657 ± 0.023	1.26×	1.58×
float	real	mutable	PropagateFast	64	10.295 ± 0.303	5.082 ± 0.073	4.613 ± 0.088	2.03×	2.23×
float	real	mutable	PropagateFast	4096	624.820 ± 14.579	427.614 ± 8.164	249.135 ± 3.360	1.46×	2.51×
float	real	mutable	PropagateFast	65536	10082.100 ± 366.615	6808.846 ± 106.811	6106.999 ± 537.172	1.48×	1.65×
float	real	mutable	PropagateNaN	3	1.878 ± 0.083	1.753 ± 0.018	1.509 ± 0.040	1.07×	1.25×
float	real	mutable	PropagateNaN	8	4.690 ± 0.118	2.519 ± 0.059	2.136 ± 0.022	1.86×	2.20×
float	real	mutable	PropagateNaN	9	5.284 ± 0.149	2.984 ± 0.027	2.686 ± 0.024	1.77×	1.97×
float	real	mutable	PropagateNaN	64	29.804 ± 0.911	9.998 ± 0.060	9.634 ± 0.104	2.98×	3.09×
float	real	mutable	PropagateNaN	4096	2002.970 ± 65.079	785.031 ± 8.193	518.985 ± 5.277	2.55×	3.86×
float	real	mutable	PropagateNaN	65536	32214.824 ± 1668.503	12641.629 ± 142.827	8372.599 ± 165.623	2.55×	3.85×
float	real	const	PropagateNaN	3	1.875 ± 0.071	1.693 ± 0.027	1.467 ± 0.016	1.11×	1.28×
float	real	const	PropagateNaN	8	4.679 ± 0.198	2.457 ± 0.006	2.116 ± 0.040	1.90×	2.21×
float	real	const	PropagateNaN	9	5.305 ± 0.193	3.026 ± 0.072	2.732 ± 0.056	1.75×	1.94×
float	real	const	PropagateNaN	64	30.942 ± 0.607	10.040 ± 0.129	10.092 ± 0.206	3.08×	3.07×
float	real	const	PropagateNaN	4096	2034.388 ± 76.699	811.135 ± 21.374	534.186 ± 12.384	2.51×	3.81×
float	real	const	PropagateNaN	65536	33151.071 ± 1317.895	12891.068 ± 248.164	8574.559 ± 326.686	2.57×	3.87×
float	imag	mutable	PropagateNaN	3	1.948 ± 0.156	1.462 ± 0.010	1.489 ± 0.017	1.33×	1.31×
float	imag	mutable	PropagateNaN	8	5.395 ± 0.175	2.465 ± 0.025	2.107 ± 0.011	2.19×	2.56×
float	imag	mutable	PropagateNaN	9	6.492 ± 0.183	3.050 ± 0.044	2.694 ± 0.028	2.13×	2.41×
float	imag	mutable	PropagateNaN	64	31.209 ± 0.996	9.806 ± 0.056	9.701 ± 0.130	3.18×	3.22×
float	imag	mutable	PropagateNaN	4096	2019.508 ± 72.999	775.370 ± 5.435	528.870 ± 6.873	2.60×	3.82×
float	imag	mutable	PropagateNaN	65536	32243.998 ± 524.854	12629.998 ± 234.368	8732.348 ± 231.698	2.55×	3.69×
float	imag	const	PropagateNaN	3	1.880 ± 0.053	1.483 ± 0.029	1.692 ± 0.017	1.27×	1.11×
float	imag	const	PropagateNaN	8	4.800 ± 0.190	2.484 ± 0.040	2.098 ± 0.021	1.93×	2.29×
float	imag	const	PropagateNaN	9	5.378 ± 0.157	3.040 ± 0.072	2.742 ± 0.069	1.77×	1.96×
float	imag	const	PropagateNaN	64	30.634 ± 0.467	10.090 ± 0.142	9.781 ± 0.192	3.04×	3.13×
float	imag	const	PropagateNaN	4096	1964.877 ± 65.215	780.182 ± 9.588	522.073 ± 3.601	2.52×	3.76×
float	imag	const	PropagateNaN	65536	32400.225 ± 816.857	12746.960 ± 203.200	8430.455 ± 310.896	2.54×	3.84×
double	real	mutable	PropagateFast	3	1.185 ± 0.030	1.482 ± 0.015	1.244 ± 0.005	0.80×	0.95×
double	real	mutable	PropagateFast	8	2.347 ± 0.023	2.280 ± 0.027	1.588 ± 0.012	1.03×	1.48×
double	real	mutable	PropagateFast	9	2.573 ± 0.069	2.682 ± 0.034	2.046 ± 0.025	0.96×	1.26×
double	real	mutable	PropagateFast	64	10.372 ± 0.446	9.018 ± 0.167	7.984 ± 0.058	1.15×	1.30×
double	real	mutable	PropagateFast	4096	653.467 ± 37.990	861.253 ± 10.562	584.826 ± 4.840	0.76×	1.12×
double	real	mutable	PropagateFast	65536	12338.598 ± 1259.960	14246.835 ± 304.546	10754.815 ± 92.478	0.87×	1.15×
double	real	mutable	PropagateNaN	3	2.248 ± 0.086	1.890 ± 0.024	1.630 ± 0.029	1.19×	1.38×
double	real	mutable	PropagateNaN	8	6.121 ± 0.304	2.895 ± 0.045	2.699 ± 0.034	2.11×	2.27×
double	real	mutable	PropagateNaN	9	7.004 ± 0.293	3.496 ± 0.035	3.198 ± 0.059	2.00×	2.19×
double	real	mutable	PropagateNaN	64	31.322 ± 0.334	18.269 ± 0.141	17.318 ± 0.395	1.71×	1.81×
double	real	mutable	PropagateNaN	4096	2041.346 ± 64.051	1612.549 ± 36.077	1054.600 ± 10.112	1.27×	1.94×
double	real	mutable	PropagateNaN	65536	33493.588 ± 1780.896	25397.186 ± 137.032	17352.673 ± 550.893	1.32×	1.93×
double	real	const	PropagateNaN	3	2.242 ± 0.083	1.819 ± 0.084	1.658 ± 0.028	1.23×	1.35×
double	real	const	PropagateNaN	8	6.227 ± 0.163	2.877 ± 0.036	2.525 ± 0.036	2.16×	2.47×
double	real	const	PropagateNaN	9	7.009 ± 0.233	3.541 ± 0.059	3.145 ± 0.032	1.98×	2.23×
double	real	const	PropagateNaN	64	30.494 ± 0.936	18.152 ± 0.172	17.387 ± 0.550	1.68×	1.75×
double	real	const	PropagateNaN	4096	2032.526 ± 56.473	1594.401 ± 19.450	1103.122 ± 17.676	1.27×	1.84×
double	real	const	PropagateNaN	65536	32502.686 ± 1673.070	25596.111 ± 246.493	17224.253 ± 349.792	1.27×	1.89×
double	imag	mutable	PropagateNaN	3	2.327 ± 0.171	1.778 ± 0.018	1.640 ± 0.012	1.31×	1.42×
double	imag	mutable	PropagateNaN	8	7.033 ± 0.177	2.899 ± 0.045	2.660 ± 0.079	2.43×	2.64×
double	imag	mutable	PropagateNaN	9	7.862 ± 0.141	3.528 ± 0.031	3.156 ± 0.016	2.23×	2.49×
double	imag	mutable	PropagateNaN	64	31.300 ± 0.423	18.069 ± 0.155	16.963 ± 0.084	1.73×	1.85×
double	imag	mutable	PropagateNaN	4096	2047.696 ± 13.913	1599.084 ± 18.075	1082.501 ± 28.651	1.28×	1.89×
double	imag	mutable	PropagateNaN	65536	32602.465 ± 1161.289	25718.262 ± 515.887	17515.949 ± 568.869	1.27×	1.86×
double	imag	const	PropagateNaN	3	2.224 ± 0.097	1.762 ± 0.024	1.636 ± 0.017	1.26×	1.36×
double	imag	const	PropagateNaN	8	6.098 ± 0.231	2.867 ± 0.024	2.499 ± 0.021	2.13×	2.44×
double	imag	const	PropagateNaN	9	7.043 ± 0.326	3.472 ± 0.038	3.139 ± 0.028	2.03×	2.24×
double	imag	const	PropagateNaN	64	30.442 ± 1.285	18.346 ± 0.442	16.950 ± 0.125	1.66×	1.80×
double	imag	const	PropagateNaN	4096	2078.047 ± 69.436	1583.480 ± 10.719	1068.625 ± 15.944	1.31×	1.94×
double	imag	const	PropagateNaN	65536	32704.772 ± 1553.655	25953.559 ± 444.998	18000.443 ± 959.512	1.26×	1.82×

GCC / AVX2

T	Component	Input	Policy	N	Original (ns)	Head (ns)	Head + !3116 (merged) (ns)	Original/head	Original/(head + !3116 (merged))
float	real	mutable	PropagateFast	3	1.045 ± 0.011	0.850 ± 0.008	0.847 ± 0.014	1.23×	1.23×
float	real	mutable	PropagateFast	8	1.871 ± 0.016	1.414 ± 0.030	1.121 ± 0.014	1.32×	1.67×
float	real	mutable	PropagateFast	9	2.099 ± 0.025	1.671 ± 0.023	1.438 ± 0.014	1.26×	1.46×
float	real	mutable	PropagateFast	64	8.967 ± 0.169	3.849 ± 0.028	3.749 ± 0.067	2.33×	2.39×
float	real	mutable	PropagateFast	4096	568.218 ± 5.094	251.005 ± 3.558	183.478 ± 2.005	2.26×	3.10×
float	real	mutable	PropagateFast	65536	9270.773 ± 294.570	5129.667 ± 32.473	5146.971 ± 333.741	1.81×	1.80×
float	real	mutable	PropagateNaN	3	1.804 ± 0.027	1.818 ± 0.042	1.791 ± 0.021	0.99×	1.01×
float	real	mutable	PropagateNaN	8	4.630 ± 0.114	2.203 ± 0.028	2.064 ± 0.020	2.10×	2.24×
float	real	mutable	PropagateNaN	9	5.278 ± 0.317	2.854 ± 0.143	2.545 ± 0.020	1.85×	2.07×
float	real	mutable	PropagateNaN	64	29.934 ± 0.643	7.811 ± 0.121	7.810 ± 0.081	3.83×	3.83×
float	real	mutable	PropagateNaN	4096	1949.107 ± 39.313	462.122 ± 5.732	321.257 ± 3.015	4.22×	6.07×
float	real	mutable	PropagateNaN	65536	30782.631 ± 528.664	7527.691 ± 117.758	5291.874 ± 44.743	4.09×	5.82×
float	real	const	PropagateNaN	3	1.816 ± 0.046	1.799 ± 0.022	1.789 ± 0.019	1.01×	1.01×
float	real	const	PropagateNaN	8	4.517 ± 0.084	2.207 ± 0.047	2.058 ± 0.022	2.05×	2.19×
float	real	const	PropagateNaN	9	5.142 ± 0.150	2.829 ± 0.060	2.564 ± 0.078	1.82×	2.01×
float	real	const	PropagateNaN	64	29.557 ± 0.826	7.853 ± 0.189	7.993 ± 0.206	3.76×	3.70×
float	real	const	PropagateNaN	4096	1929.110 ± 35.275	462.002 ± 7.673	327.903 ± 4.483	4.18×	5.88×
float	real	const	PropagateNaN	65536	30776.899 ± 385.373	7604.176 ± 80.722	5257.780 ± 75.380	4.05×	5.85×
float	imag	mutable	PropagateNaN	3	1.842 ± 0.062	1.812 ± 0.035	1.798 ± 0.024	1.02×	1.02×
float	imag	mutable	PropagateNaN	8	5.234 ± 0.090	2.158 ± 0.026	2.061 ± 0.033	2.43×	2.54×
float	imag	mutable	PropagateNaN	9	6.158 ± 0.067	2.775 ± 0.053	2.546 ± 0.012	2.22×	2.42×
float	imag	mutable	PropagateNaN	64	29.268 ± 0.223	7.506 ± 0.113	7.776 ± 0.150	3.90×	3.76×
float	imag	mutable	PropagateNaN	4096	1954.030 ± 64.089	460.839 ± 8.310	320.063 ± 7.043	4.24×	6.11×
float	imag	mutable	PropagateNaN	65536	31728.105 ± 522.147	7376.573 ± 104.003	5115.997 ± 38.241	4.30×	6.20×
float	imag	const	PropagateNaN	3	1.814 ± 0.036	1.811 ± 0.027	1.789 ± 0.017	1.00×	1.01×
float	imag	const	PropagateNaN	8	4.470 ± 0.040	2.184 ± 0.071	2.066 ± 0.024	2.05×	2.16×
float	imag	const	PropagateNaN	9	5.257 ± 0.245	2.781 ± 0.026	2.536 ± 0.037	1.89×	2.07×
float	imag	const	PropagateNaN	64	29.003 ± 0.226	7.868 ± 0.203	7.777 ± 0.056	3.69×	3.73×
float	imag	const	PropagateNaN	4096	1899.708 ± 18.208	477.629 ± 19.476	312.639 ± 2.990	3.98×	6.08×
float	imag	const	PropagateNaN	65536	31092.202 ± 809.497	7564.972 ± 111.944	5467.967 ± 395.449	4.11×	5.69×
double	real	mutable	PropagateFast	3	1.341 ± 0.144	0.836 ± 0.013	0.810 ± 0.010	1.60×	1.66×
double	real	mutable	PropagateFast	8	2.355 ± 0.156	1.821 ± 0.036	1.660 ± 0.029	1.29×	1.42×
double	real	mutable	PropagateFast	9	2.656 ± 0.191	2.307 ± 0.046	2.004 ± 0.028	1.15×	1.33×
double	real	mutable	PropagateFast	64	8.803 ± 0.106	6.075 ± 0.296	6.267 ± 0.035	1.45×	1.40×
double	real	mutable	PropagateFast	4096	588.999 ± 4.768	632.204 ± 3.168	594.253 ± 14.509	0.93×	0.99×
double	real	mutable	PropagateFast	65536	10922.796 ± 683.059	11517.128 ± 128.536	11019.114 ± 233.457	0.95×	0.99×
double	real	mutable	PropagateNaN	3	2.173 ± 0.063	1.701 ± 0.030	1.676 ± 0.010	1.28×	1.30×
double	real	mutable	PropagateNaN	8	5.852 ± 0.045	2.598 ± 0.063	2.207 ± 0.025	2.25×	2.65×
double	real	mutable	PropagateNaN	9	6.611 ± 0.062	3.825 ± 0.135	3.466 ± 0.069	1.73×	1.91×
double	real	mutable	PropagateNaN	64	29.555 ± 0.555	12.231 ± 0.109	11.941 ± 0.325	2.42×	2.48×
double	real	mutable	PropagateNaN	4096	1900.823 ± 13.157	929.114 ± 13.787	713.554 ± 60.386	2.05×	2.66×
double	real	mutable	PropagateNaN	65536	30827.705 ± 540.198	15281.037 ± 550.086	14225.942 ± 829.675	2.02×	2.17×
double	real	const	PropagateNaN	3	2.155 ± 0.024	1.686 ± 0.023	1.676 ± 0.014	1.28×	1.29×
double	real	const	PropagateNaN	8	6.054 ± 0.144	2.553 ± 0.028	2.195 ± 0.019	2.37×	2.76×
double	real	const	PropagateNaN	9	6.662 ± 0.047	3.693 ± 0.071	3.444 ± 0.044	1.80×	1.93×
double	real	const	PropagateNaN	64	29.062 ± 0.202	12.291 ± 0.249	11.720 ± 0.411	2.36×	2.48×
double	real	const	PropagateNaN	4096	1956.280 ± 31.715	960.304 ± 21.976	726.373 ± 57.139	2.04×	2.69×
double	real	const	PropagateNaN	65536	31989.571 ± 1186.153	15335.348 ± 429.614	12835.038 ± 1929.225	2.09×	2.49×
double	imag	mutable	PropagateNaN	3	2.158 ± 0.043	1.718 ± 0.047	1.699 ± 0.040	1.26×	1.27×
double	imag	mutable	PropagateNaN	8	6.822 ± 0.122	2.527 ± 0.023	2.191 ± 0.022	2.70×	3.11×
double	imag	mutable	PropagateNaN	9	7.753 ± 0.046	3.639 ± 0.091	3.427 ± 0.034	2.13×	2.26×
double	imag	mutable	PropagateNaN	64	29.508 ± 0.258	12.036 ± 0.120	11.325 ± 0.103	2.45×	2.61×
double	imag	mutable	PropagateNaN	4096	1921.000 ± 30.249	906.100 ± 4.018	696.712 ± 27.273	2.12×	2.76×
double	imag	mutable	PropagateNaN	65536	31112.539 ± 608.689	14681.187 ± 132.212	11391.643 ± 881.170	2.12×	2.73×
double	imag	const	PropagateNaN	3	2.162 ± 0.045	1.690 ± 0.015	1.697 ± 0.021	1.28×	1.27×
double	imag	const	PropagateNaN	8	5.893 ± 0.065	2.624 ± 0.040	2.191 ± 0.017	2.25×	2.69×
double	imag	const	PropagateNaN	9	6.653 ± 0.078	3.778 ± 0.101	3.418 ± 0.058	1.76×	1.95×
double	imag	const	PropagateNaN	64	29.370 ± 0.550	12.123 ± 0.385	11.497 ± 0.159	2.42×	2.55×
double	imag	const	PropagateNaN	4096	1968.708 ± 40.374	912.512 ± 11.506	706.906 ± 30.193	2.16×	2.78×
double	imag	const	PropagateNaN	65536	30907.478 ± 335.522	15776.921 ± 751.486	14329.185 ± 339.685	1.96×	2.16×

Clang / AVX2

T	Component	Input	Policy	N	Original (ns)	Head (ns)	Head + !3116 (merged) (ns)	Original/head	Original/(head + !3116 (merged))
float	real	mutable	PropagateFast	3	1.292 ± 0.016	1.601 ± 0.038	1.387 ± 0.024	0.81×	0.93×
float	real	mutable	PropagateFast	8	2.073 ± 0.021	1.380 ± 0.030	1.139 ± 0.017	1.50×	1.82×
float	real	mutable	PropagateFast	9	2.048 ± 0.025	1.705 ± 0.060	1.461 ± 0.028	1.20×	1.40×
float	real	mutable	PropagateFast	64	12.490 ± 0.260	3.394 ± 0.124	3.151 ± 0.039	3.68×	3.96×
float	real	mutable	PropagateFast	4096	743.729 ± 11.896	245.494 ± 5.095	159.164 ± 2.061	3.03×	4.67×
float	real	mutable	PropagateFast	65536	11695.462 ± 108.614	4059.037 ± 56.759	3288.800 ± 25.310	2.88×	3.56×
float	real	mutable	PropagateNaN	3	1.862 ± 0.040	2.495 ± 0.045	2.467 ± 0.027	0.75×	0.75×
float	real	mutable	PropagateNaN	8	5.464 ± 0.073	2.157 ± 0.045	2.062 ± 0.032	2.53×	2.65×
float	real	mutable	PropagateNaN	9	7.034 ± 0.079	3.349 ± 0.157	3.194 ± 0.098	2.10×	2.20×
float	real	mutable	PropagateNaN	64	24.812 ± 0.313	6.991 ± 0.148	7.130 ± 0.085	3.55×	3.48×
float	real	mutable	PropagateNaN	4096	1635.576 ± 30.082	450.771 ± 6.473	307.682 ± 2.599	3.63×	5.32×
float	real	mutable	PropagateNaN	65536	26816.896 ± 307.557	7282.740 ± 46.801	5198.744 ± 164.017	3.68×	5.16×
float	real	const	PropagateNaN	3	1.866 ± 0.022	2.503 ± 0.030	2.525 ± 0.043	0.75×	0.74×
float	real	const	PropagateNaN	8	5.398 ± 0.029	2.249 ± 0.029	2.087 ± 0.026	2.40×	2.59×
float	real	const	PropagateNaN	9	6.960 ± 0.067	3.236 ± 0.060	3.074 ± 0.057	2.15×	2.26×
float	real	const	PropagateNaN	64	24.826 ± 0.282	6.939 ± 0.143	7.149 ± 0.093	3.58×	3.47×
float	real	const	PropagateNaN	4096	1661.309 ± 45.915	462.790 ± 8.259	313.783 ± 8.240	3.59×	5.29×
float	real	const	PropagateNaN	65536	26314.988 ± 522.781	7374.290 ± 100.374	5300.719 ± 255.280	3.57×	4.96×
float	imag	mutable	PropagateNaN	3	1.876 ± 0.048	2.658 ± 0.030	2.520 ± 0.032	0.71×	0.74×
float	imag	mutable	PropagateNaN	8	5.302 ± 0.015	2.250 ± 0.027	2.122 ± 0.049	2.36×	2.50×
float	imag	mutable	PropagateNaN	9	6.173 ± 0.051	3.145 ± 0.034	3.079 ± 0.034	1.96×	2.00×
float	imag	mutable	PropagateNaN	64	25.297 ± 0.369	7.098 ± 0.259	7.284 ± 0.113	3.56×	3.47×
float	imag	mutable	PropagateNaN	4096	1662.951 ± 35.972	475.237 ± 9.942	309.380 ± 4.148	3.50×	5.38×
float	imag	mutable	PropagateNaN	65536	26858.125 ± 468.957	7474.999 ± 211.357	5149.296 ± 185.673	3.59×	5.22×
float	imag	const	PropagateNaN	3	1.868 ± 0.025	2.496 ± 0.018	2.471 ± 0.031	0.75×	0.76×
float	imag	const	PropagateNaN	8	5.263 ± 0.024	2.187 ± 0.033	2.016 ± 0.020	2.41×	2.61×
float	imag	const	PropagateNaN	9	7.219 ± 0.049	3.214 ± 0.055	3.129 ± 0.072	2.25×	2.31×
float	imag	const	PropagateNaN	64	25.626 ± 0.599	7.027 ± 0.066	7.226 ± 0.119	3.65×	3.55×
float	imag	const	PropagateNaN	4096	1666.937 ± 49.021	473.572 ± 13.636	308.081 ± 5.554	3.52×	5.41×
float	imag	const	PropagateNaN	65536	26270.465 ± 247.947	7499.698 ± 147.810	5078.874 ± 97.115	3.50×	5.17×
double	real	mutable	PropagateFast	3	1.266 ± 0.024	1.553 ± 0.013	1.840 ± 0.016	0.82×	0.69×
double	real	mutable	PropagateFast	8	2.324 ± 0.031	1.641 ± 0.017	2.277 ± 0.042	1.42×	1.02×
double	real	mutable	PropagateFast	9	2.500 ± 0.035	2.086 ± 0.035	2.504 ± 0.048	1.20×	1.00×
double	real	mutable	PropagateFast	64	11.162 ± 0.115	7.389 ± 0.144	7.406 ± 0.107	1.51×	1.51×
double	real	mutable	PropagateFast	4096	654.908 ± 25.421	530.481 ± 4.020	462.278 ± 7.479	1.23×	1.42×
double	real	mutable	PropagateFast	65536	11011.593 ± 188.140	9501.072 ± 365.922	9072.209 ± 410.660	1.16×	1.21×
double	real	mutable	PropagateNaN	3	1.833 ± 0.016	2.462 ± 0.025	2.329 ± 0.059	0.74×	0.79×
double	real	mutable	PropagateNaN	8	4.691 ± 0.072	2.713 ± 0.054	2.750 ± 0.053	1.73×	1.71×
double	real	mutable	PropagateNaN	9	5.576 ± 0.080	3.624 ± 0.039	3.806 ± 0.153	1.54×	1.47×
double	real	mutable	PropagateNaN	64	30.324 ± 0.515	13.437 ± 0.250	13.406 ± 0.327	2.26×	2.26×
double	real	mutable	PropagateNaN	4096	1796.160 ± 22.060	962.157 ± 15.015	755.777 ± 8.710	1.87×	2.38×
double	real	mutable	PropagateNaN	65536	30471.752 ± 570.174	15460.074 ± 193.094	12795.787 ± 533.919	1.97×	2.38×
double	real	const	PropagateNaN	3	1.833 ± 0.009	2.513 ± 0.058	2.288 ± 0.023	0.73×	0.80×
double	real	const	PropagateNaN	8	4.652 ± 0.072	2.736 ± 0.054	2.765 ± 0.061	1.70×	1.68×
double	real	const	PropagateNaN	9	5.512 ± 0.083	3.720 ± 0.123	3.993 ± 0.102	1.48×	1.38×
double	real	const	PropagateNaN	64	30.226 ± 0.220	13.692 ± 0.306	13.356 ± 0.231	2.21×	2.26×
double	real	const	PropagateNaN	4096	1809.544 ± 35.486	993.310 ± 39.647	738.333 ± 7.394	1.82×	2.45×
double	real	const	PropagateNaN	65536	29806.413 ± 588.702	15342.487 ± 150.844	12251.203 ± 163.693	1.94×	2.43×
double	imag	mutable	PropagateNaN	3	1.835 ± 0.011	2.527 ± 0.055	2.277 ± 0.024	0.73×	0.81×
double	imag	mutable	PropagateNaN	8	4.715 ± 0.060	2.709 ± 0.016	2.743 ± 0.097	1.74×	1.72×
double	imag	mutable	PropagateNaN	9	5.554 ± 0.151	3.797 ± 0.043	3.816 ± 0.082	1.46×	1.46×
double	imag	mutable	PropagateNaN	64	30.616 ± 0.633	13.560 ± 0.130	13.184 ± 0.225	2.26×	2.32×
double	imag	mutable	PropagateNaN	4096	1800.310 ± 28.306	962.311 ± 14.011	757.411 ± 6.030	1.87×	2.38×
double	imag	mutable	PropagateNaN	65536	29886.644 ± 896.212	15452.569 ± 288.577	14246.811 ± 581.283	1.93×	2.10×
double	imag	const	PropagateNaN	3	1.856 ± 0.022	2.528 ± 0.065	2.283 ± 0.024	0.73×	0.81×
double	imag	const	PropagateNaN	8	4.653 ± 0.088	2.674 ± 0.023	2.715 ± 0.078	1.74×	1.71×
double	imag	const	PropagateNaN	9	5.644 ± 0.090	3.770 ± 0.036	3.826 ± 0.102	1.50×	1.48×
double	imag	const	PropagateNaN	64	30.831 ± 1.071	13.262 ± 0.209	13.579 ± 0.295	2.32×	2.27×
double	imag	const	PropagateNaN	4096	1790.195 ± 19.692	947.790 ± 7.874	757.026 ± 18.622	1.89×	2.36×
double	imag	const	PropagateNaN	65536	29078.432 ± 794.130	15735.387 ± 305.408	12618.922 ± 653.016	1.85×	2.30×

Valgrind Callgrind: 100 reductions of 4,096 components

GCC 13.3, C++14, -O3 -g -DNDEBUG, the ISA flags above, Valgrind 3.22. The driver warms up and validates first, enables instrumentation only around 100 reduction calls, and verifies checksum 5000. Counts include the same loop/control overhead in both revisions. These are collected instruction counts (totals: Ir), not elapsed timings.

valgrind --tool=callgrind --instr-atstart=no --callgrind-out-file=counts.out \
  ./callgrind-driver CASE
# CASE: 0 float/Fast, 1 float/NaN, 2 double/Fast, 3 double/NaN

ISA	T	Policy	Previous instructions	Current instructions	Reduction
SSE	float	PropagateFast	1,232,816	772,016	37.4%
SSE	float	PropagateNaN	1,848,316	1,387,416	24.9%
SSE	double	PropagateFast	1,437,616	1,437,616	0.0%
SSE	double	PropagateNaN	2,666,316	2,666,316	0.0%
AVX	float	PropagateFast	772,116	465,016	39.8%
AVX	float	PropagateNaN	875,116	567,916	35.1%
AVX	double	PropagateFast	925,616	925,616	0.0%
AVX	double	PropagateNaN	1,130,716	1,130,716	0.0%
AVX2	float	PropagateFast	772,116	336,916	56.4%
AVX2	float	PropagateNaN	875,216	439,916	49.7%
AVX2	double	PropagateFast	925,616	720,716	22.1%
AVX2	double	PropagateNaN	1,130,816	925,916	18.1%

Raw measurements, test logs, Callgrind counts, and assembly: issue3097-stride2-evidence.zip

Edited Sep 17, 2026 by Rasmus Munk Larsen

Core: Vectorize complex component reductions

Double PropagateFast and !3116 (merged)

Reference issue

Description

Validation

Merge request reports

Double `PropagateFast` and !3116 (merged)