Add runtime cache size detection for ARM and improve GEMM blocking

Summary

Previously, queryCacheSizes() only worked on x86 (via CPUID). All other architectures returned -1 and fell back to very conservative defaults (L1=16KB, L2=512KB, L3=512KB), significantly underestimating modern ARM caches and causing suboptimal GEMM blocking parameters.

  • Add macOS cache detection via sysctlbyname. Prefers P-core L1 size (hw.perflevel0.l1dcachesize) since compute-heavy work like GEMM runs on performance cores. Uses conservative hw.l2cachesize for L2 (the P-core L2 is shared among all P-cores and would overestimate per-core capacity).
  • Add Linux/POSIX cache detection via sysconf(_SC_LEVEL1_DCACHE_SIZE) etc.
  • Add ARM-specific default fallbacks (L1=64KB, L2=1MB, L3=4MB) for when runtime detection is unavailable.
  • Fix max_nc computation in evaluateProductBlockingSizesHeuristic to use actual kc rather than max_kc, preventing unnecessary nc squeeze on CPUs with large L1 caches.
  • Add 4096 size to GEMM benchmark.

Benchmark results

Single-precision float GEMM on Apple M4 (10 cores, macOS 15.4, Apple Clang 17):

Size Before (GFLOPS) After (GFLOPS) Change
64 105.3 104.4 -0.9%
256 116.1 117.2 +0.9%
512 119.6 122.2 +2.2%
768 121.1 123.0 +1.6%
1024 120.4 122.6 +1.8%
1536 121.0 123.6 +2.1%
2048 121.6 123.5 +1.6%
4096 — 122.7 (new)

Detected cache sizes on Apple M4 (vs old defaults):

  • L1: 128KB P-core (was 16KB default — 8x underestimate)
  • L2: 4MB (was 512KB default — 8x underestimate)
  • L3: falls back to 4MB ARM default (Apple Silicon has no traditional L3)

Test plan

  • All product_large, product_extra, product_small tests pass (46/46)
  • Benchmarked on Apple M4 (results above)
  • Verify no regression on x86 (CPUID path unchanged)
  • Verify Linux ARM path compiles and detects cache sizes correctly

🤖 Generated with Claude Code

Edited by Rasmus Munk Larsen

Merge request reports

Loading
Loading