## Goal Publish current, provenance-tagged performance results for Eigen against other BLAS/LAPACK libraries on the website, and retire the 2009–2011 material there and in `doc/`. Everything published dates from 2009–2011 and was produced with tooling (BTL, `bench/perf_monitoring/`) that no longer exists in the tree; a reader following the site's own instructions today cannot regenerate any of it. Status as of 2026-09-10: the first dataset exists (Zen 5, three libraries, three operations) and the website change is prepared; see the progress log in the comments. ## Approach (decided 2026-08-20, comments below) - **Cross-library comparison is a first-class published result**: Eigen against each library as a peer, keyed by BLAS/LAPACK mnemonic with the Eigen spelling beside it. The `EIGEN_USE_BLAS` "does enabling the backend help" arm is a secondary output. - **Generation is off-CI, per release, on owned hardware.** The benchmark CI pipeline never produced a result (#3111) and has since been removed (!3026); nothing here depends on CI. Continuous regression tracking is a separate topic (#3146). - **Numbers live only on the website, with a provenance block** (CPU, OS, compiler and flags, Eigen commit, library versions, thread count, method, date). The manual carries qualitative, size-qualified guidance plus a dated link, because it ships with the release and cannot be corrected afterwards. - **Single-threaded is the headline**, with every library's thread cap set explicitly; thread scaling is a separate sweep. Size grids span tiny (2–16), small (24–128), medium (192–1024) and large (1536–8192) plus non-power-of-two aliasing probes and tall-skinny shapes, because the ranking inverts with size. - Libraries: OpenBLAS everywhere (the one common arm); Intel oneMKL, AMD AOCL-BLIS/libFLAME and netlib on x86-64; NVPL and Arm Performance Libraries on aarch64; Accelerate on macOS. Platforms: x86-64 across SSE2/AVX2/AVX-512, aarch64 NEON and SVE2, macOS. POWER, s390x, MSA, LoongArch, RISC-V and Windows/MSVC are explicitly out of scope. ## What exists today ### Website — <https://libeigen.gitlab.io/benchmarks/>, source <https://gitlab.com/libeigen/libeigen.gitlab.io> | Page | Dated | Hardware | Eigen measured | |---|---|---|---| | Benchmark | 23 March 2011 | Core 2 Quad Q9400 @ 2.66GHz, gcc 4.5.0 | eigen3 (SSE2), eigen2 | | Benchmark Intel | 13 February 2009 | Pentium D 3.00GHz (Prescott) | eigen2 r939062 | | Benchmark AMD | 13 February 2009 | Athlon 64 X2 5000+ | eigen2 r939062 | | Benchmark AMD Intel compare | 17 February 2009 | both of the above | eigen2 r939062 | Comparison libraries MKL 10.1/11.0, GotoBLAS 1.26/2-1.13, ATLAS 3.8.3, ACML 4.2.0; everything SSE2/SSE3 and single-threaded. `/pages/how_to_run_the_benchmark_suite` instructs `-DEIGEN_BUILD_BTL=ON` and `/pages/performance_monitoring` instructs `script/eigen_monitor_perf.sh`; both were removed in a95440de1 (!2217). ### Manual — `doc/` - `DenseDecompositionBenchmark.dox` — the only quantitative table; provenance "a laptop equipped with an Intel core i7 @ 2,6 GHz", no version, compiler or date. - `HiPerformance.dox`, `TopicVectorization.dox`, `TopicMultithreading.dox` — advice with no measurement behind it; the only speedup figure in the manual is "up to 4x faster" for SSE2 in `InsideEigenExample.dox`. - `TopicLinearAlgebraDecompositions.dox`, `TutorialLinearAlgebra.dox` — size-independent Speed columns; `RandColPivHouseholderQR` rated the same as `ColPivHouseholderQR` although its purpose is to scale differently; `RandCompleteOrthogonalDecomposition` has no row. - `UsingIntelMKL.dox`, `UsingAOCL.dox`, `UsingBlasLapackBackends.dox` — how to enable a backend, never when it pays. ### Tooling `benchmarks/` builds 60 Google Benchmark executables across 14 modules (Eigen against itself). The cross-library comparison benchmarks are `benchmarks/comparison/` on the branch of !2903, stripped on 2026-09-10 to what producing a page took: one binary per reference library carrying an Eigen arm and a library arm on the same operands, each library result validated against Eigen before timing, one flop formula per operation shared with the rest of `benchmarks/` through `bench_common.h`, a vendor table, machine profiles, and a `run.py` that builds, drives and writes one result file per run with the machine, build, library version and run conditions. GEMM, GEMV and POTRF are implemented. The website repository's `scripts/benchmarks/plot_comparison.py` draws the pages from the result files. ## Work items 1. ~~Unblock the CI generator (#3111).~~ Superseded: generation is off-CI; the pipeline was removed. 2. **Website: retire the legacy material and publish from the harness.** The first page (Zen 5, September 2026) with the index restructured into current and historical results, the how-to page rewritten for the Google Benchmark suite, and the performance-monitoring page marked historical is prepared on a branch of the website repository; merge request to follow. 3. **Land the comparison benchmarks in Eigen.** !2903 is now the minimum (29 files, +2.7k lines): `bench_common.h` with the flop-count corrections, the three arms, the vendor table, machine profiles and the 370-line `run.py`; rendering lives beside the data in the website repository. 4. **Extend coverage**: more operations (TRSM, GETRF, GEQRF, GESDD, SYEV first, then the Eigen-only decompositions with "no LAPACK counterpart" rows), complex scalars, the AVX2 and SSE2 targets, a thread-scaling sweep, and more machines (Intel for an uncontested MKL datapoint, aarch64 NEON and SVE2, Apple). 5. **Regenerate `DenseDecompositionBenchmark`** once the decomposition operations are in, after extending `bench_dense_solvers` to the randomized QR/COD classes and BunchKaufman. 6. **Give the qualitative pages real content**: size-qualified Speed columns where the ranking inverts; measured per-ISA payoff for `TopicVectorization`; scaling data for `TopicMultithreading`; measured guidance on when `EIGEN_USE_BLAS` / `EIGEN_USE_LAPACKE` helps. ## Context Surfaced during the documentation pass (!2839) giving `CompleteOrthogonalDecomposition` and `RandCompleteOrthogonalDecomposition` proper coverage: the speed claims there were kept algorithmic because no trustworthy measurement was available.