Loading
Add runtime cache size detection for ARM and improve GEMM blocking
Summary
Previously, queryCacheSizes() only worked on x86 (via CPUID). All other architectures returned -1 and fell back to very conservative defaults (L1=16KB, L2=512KB, L3=512KB), significantly underestimating modern ARM caches and causing suboptimal GEMM blocking parameters.
- Add macOS cache detection via
sysctlbyname. Prefers P-core L1 size (hw.perflevel0.l1dcachesize) since compute-heavy work like GEMM runs on performance cores. Uses conservativehw.l2cachesizefor L2 (the P-core L2 is shared among all P-cores and would overestimate per-core capacity). - Add Linux/POSIX cache detection via
sysconf(_SC_LEVEL1_DCACHE_SIZE)etc. - Add ARM-specific default fallbacks (L1=64KB, L2=1MB, L3=4MB) for when runtime detection is unavailable.
- Fix
max_nccomputation inevaluateProductBlockingSizesHeuristicto use actualkcrather thanmax_kc, preventing unnecessaryncsqueeze on CPUs with large L1 caches. - Add 4096 size to GEMM benchmark.
Benchmark results
Single-precision float GEMM on Apple M4 (10 cores, macOS 15.4, Apple Clang 17):
| Size | Before (GFLOPS) | After (GFLOPS) | Change |
|---|---|---|---|
| 64 | 105.3 | 104.4 | -0.9% |
| 256 | 116.1 | 117.2 | +0.9% |
| 512 | 119.6 | 122.2 | +2.2% |
| 768 | 121.1 | 123.0 | +1.6% |
| 1024 | 120.4 | 122.6 | +1.8% |
| 1536 | 121.0 | 123.6 | +2.1% |
| 2048 | 121.6 | 123.5 | +1.6% |
| 4096 | — | 122.7 | (new) |
Detected cache sizes on Apple M4 (vs old defaults):
- L1: 128KB P-core (was 16KB default — 8x underestimate)
- L2: 4MB (was 512KB default — 8x underestimate)
- L3: falls back to 4MB ARM default (Apple Silicon has no traditional L3)
Test plan
- All
product_large,product_extra,product_smalltests pass (46/46) - Benchmarked on Apple M4 (results above)
- Verify no regression on x86 (CPUID path unchanged)
- Verify Linux ARM path compiles and detects cache sizes correctly
Edited by Rasmus Munk Larsen