Doc: replace the 2009-2011 benchmark material in doc/ and on the website with data from the Google Benchmark suite

Goal

Publish current, provenance-tagged performance results for Eigen against other BLAS/LAPACK libraries on the website, and retire the 2009–2011 material there and in doc/. Everything published dates from 2009–2011 and was produced with tooling (BTL, bench/perf_monitoring/) that no longer exists in the tree; a reader following the site's own instructions today cannot regenerate any of it.

Status as of 2026-09-10: the first dataset exists (Zen 5, three libraries, three operations) and the website change is prepared; see the progress log in the comments.

Approach (decided 2026-08-20, comments below)

  • Cross-library comparison is a first-class published result: Eigen against each library as a peer, keyed by BLAS/LAPACK mnemonic with the Eigen spelling beside it. The EIGEN_USE_BLAS "does enabling the backend help" arm is a secondary output.
  • Generation is off-CI, per release, on owned hardware. The benchmark CI pipeline never produced a result (#3111 (closed)) and has since been removed (!3026 (merged)); nothing here depends on CI. Continuous regression tracking is a separate topic (#3146).
  • Numbers live only on the website, with a provenance block (CPU, OS, compiler and flags, Eigen commit, library versions, thread count, method, date). The manual carries qualitative, size-qualified guidance plus a dated link, because it ships with the release and cannot be corrected afterwards.
  • Single-threaded is the headline, with every library's thread cap set explicitly; thread scaling is a separate sweep. Size grids span tiny (2–16), small (24–128), medium (192–1024) and large (1536–8192) plus non-power-of-two aliasing probes and tall-skinny shapes, because the ranking inverts with size.
  • Libraries: OpenBLAS everywhere (the one common arm); Intel oneMKL, AMD AOCL-BLIS/libFLAME and netlib on x86-64; NVPL and Arm Performance Libraries on aarch64; Accelerate on macOS. Platforms: x86-64 across SSE2/AVX2/AVX-512, aarch64 NEON and SVE2, macOS. POWER, s390x, MSA, LoongArch, RISC-V and Windows/MSVC are explicitly out of scope.

What exists today

Website — https://libeigen.gitlab.io/benchmarks/, source https://gitlab.com/libeigen/libeigen.gitlab.io

Page Dated Hardware Eigen measured
Benchmark 23 March 2011 Core 2 Quad Q9400 @ 2.66GHz, gcc 4.5.0 eigen3 (SSE2), eigen2
Benchmark Intel 13 February 2009 Pentium D 3.00GHz (Prescott) eigen2 r939062
Benchmark AMD 13 February 2009 Athlon 64 X2 5000+ eigen2 r939062
Benchmark AMD Intel compare 17 February 2009 both of the above eigen2 r939062

Comparison libraries MKL 10.1/11.0, GotoBLAS 1.26/2-1.13, ATLAS 3.8.3, ACML 4.2.0; everything SSE2/SSE3 and single-threaded. /pages/how_to_run_the_benchmark_suite instructs -DEIGEN_BUILD_BTL=ON and /pages/performance_monitoring instructs script/eigen_monitor_perf.sh; both were removed in a95440de (!2217 (merged)).

Manual — doc/

  • DenseDecompositionBenchmark.dox — the only quantitative table; provenance "a laptop equipped with an Intel core i7 @ 2,6 GHz", no version, compiler or date.
  • HiPerformance.dox, TopicVectorization.dox, TopicMultithreading.dox — advice with no measurement behind it; the only speedup figure in the manual is "up to 4x faster" for SSE2 in InsideEigenExample.dox.
  • TopicLinearAlgebraDecompositions.dox, TutorialLinearAlgebra.dox — size-independent Speed columns; RandColPivHouseholderQR rated the same as ColPivHouseholderQR although its purpose is to scale differently; RandCompleteOrthogonalDecomposition has no row.
  • UsingIntelMKL.dox, UsingAOCL.dox, UsingBlasLapackBackends.dox — how to enable a backend, never when it pays.

Tooling

benchmarks/ builds 60 Google Benchmark executables across 14 modules (Eigen against itself). The cross-library comparison benchmarks are benchmarks/comparison/ on the branch of !2903, stripped on 2026-09-10 to what producing a page took: one binary per reference library carrying an Eigen arm and a library arm on the same operands, each library result validated against Eigen before timing, one flop formula per operation shared with the rest of benchmarks/ through bench_common.h, a vendor table, machine profiles, and a run.py that builds, drives and writes one result file per run with the machine, build, library version and run conditions. GEMM, GEMV and POTRF are implemented. The website repository's scripts/benchmarks/plot_comparison.py draws the pages from the result files.

Work items

  1. Unblock the CI generator (#3111 (closed)). Superseded: generation is off-CI; the pipeline was removed.
  2. Website: retire the legacy material and publish from the harness. The first page (Zen 5, September 2026) with the index restructured into current and historical results, the how-to page rewritten for the Google Benchmark suite, and the performance-monitoring page marked historical is prepared on a branch of the website repository; merge request to follow.
  3. Land the comparison benchmarks in Eigen. !2903 is now the minimum (29 files, +2.7k lines): bench_common.h with the flop-count corrections, the three arms, the vendor table, machine profiles and the 370-line run.py; rendering lives beside the data in the website repository.
  4. Extend coverage: more operations (TRSM, GETRF, GEQRF, GESDD, SYEV first, then the Eigen-only decompositions with "no LAPACK counterpart" rows), complex scalars, the AVX2 and SSE2 targets, a thread-scaling sweep, and more machines (Intel for an uncontested MKL datapoint, aarch64 NEON and SVE2, Apple).
  5. Regenerate DenseDecompositionBenchmark once the decomposition operations are in, after extending bench_dense_solvers to the randomized QR/COD classes and BunchKaufman.
  6. Give the qualitative pages real content: size-qualified Speed columns where the ranking inverts; measured per-ISA payoff for TopicVectorization; scaling data for TopicMultithreading; measured guidance on when EIGEN_USE_BLAS / EIGEN_USE_LAPACKE helps.

Context

Surfaced during the documentation pass (!2839 (merged)) giving CompleteOrthogonalDecomposition and RandCompleteOrthogonalDecomposition proper coverage: the speed claims there were kept algorithmic because no trustworthy measurement was available.

Edited by Rasmus Munk Larsen