Commit 59ec397b authored by cznic's avatar cznic
Browse files

doc.go: add a Performance section with measured driver-vs-C ratios

Prompted by https://github.com/coderage-labs/spillway/issues/162, which
attributed a 103 s query to this driver spilling a temp B-tree in Go. Running
that workload here showed no spill (the LIMIT-bounded ephemeral b-tree path
never touches a temp file) and a CPU cost of about 2x the same C build, the
103 s being unbounded overlap of periodic polls.

The section records what was measured: CPU time per query on linux/amd64
against SQLite 3.53.4 compiled with our options, 2.0x/1.9x/1.3x with
modernc.org/libc v1.75.7 and 3.0x/2.2x/1.6x with the transpiled musl mem*
routines of earlier libc versions; that the ratios hold under concurrency;
and the two practical consequences, indexing what ORDER BY/GROUP BY/WHERE
use and bounding the database/sql pool.

Co-Authored-By: default avatarClaude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PmfDBCBKB4vdyiiG94dXK9
parent 51cbcbff
Loading
Loading
Loading
Loading
+45 −0
Changes for doc.go: 45 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -78,6 +78,51 @@
//
// [The SQLite Drivers Benchmarks Game]
//
// # Performance
//
// The transpiled SQLite core runs slower than the same C compiled natively.
// The gap is in CPU-bound work: the bytecode interpreter loop, b-tree page
// balancing and record building. I/O-bound work is dominated by the operating
// system either way. The ratios below are CPU time per query, measured in
// September 2026 on linux/amd64 with Go 1.27 and modernc.org/libc v1.75.7,
// against SQLite 3.53.4 compiled with the same compile-time options this
// package uses:
//
//	Workload                                                        Driver vs C
//	-------------------------------------------------------------------------
//	Unindexed ORDER BY ... LIMIT 100 over 584k rows of 23 columns      2.0x
//	GROUP BY aggregate over the same table                             1.9x
//	Correlated subquery walking an index with text comparisons         1.3x
//
// Throughput across four connections scaled at least as well as the C build
// did, so the ratios hold under concurrency.
//
// Two things follow. First, this package uses the same query planner as C
// SQLite, so a query that is slow in C is slower here by the ratio above and
// no more; but a missing index costs the same ratio more, and a query that is
// merely sluggish in C can cross a deadline here. Check EXPLAIN QUERY PLAN for
// USE TEMP B-TREE and index the columns that ORDER BY, GROUP BY and WHERE use.
// Second, database/sql opens connections without limit by default. Each
// connection carries its own page cache and its own libc thread state, and a
// periodic query that takes longer than its period piles up without bound.
// Bound the pool with [sql.DB.SetMaxOpenConns] and do not issue a periodic
// query before the previous one has returned.
//
// Part of the gap is in modernc.org/libc rather than in the transpiled SQLite.
// On Linux, libc versions before v1.75.7 implemented memcpy, memmove, memset
// and memcmp as transpiled musl loops moving at most four bytes per step;
// v1.75.7 replaced them with native Go routines backed by the runtime's
// vectorized memmove, and the three workloads above went from 3.0x, 2.2x and
// 1.6x to the figures shown. The libc version this package is validated
// against is the one pinned in its go.mod, see "Fragile modernc.org/libc
// dependency" above. On the non-Linux targets memcpy and memmove are native Go
// copies already; memcmp there is still a byte loop.
//
// Because everything is Go, the usual Go tooling reaches into the SQLite core:
// a CPU profile taken with runtime/pprof attributes time to the transpiled
// SQLite functions under their C names, for example lib._balance_nonroot or
// lib.Xsqlite3_step, and to the libc routines they call.
//
// # Builders
//
// Builder results available at: