docs: mkdocs site + per-operation driver benchmark suite

Summary

Adds a GitLab Pages docs site for django-vpg and a driver-level A/B benchmark that measures the specific things this driver does differently from psycopg.

The docs site follows the same mkdocs-material setup as django-vcache and django-vtasks. Nine pages, deployed on main. site_url is set to django-vpg.glitchtip.com — the Pages domain still needs pointing. Worth noting the siblings are inconsistent here (vcache.glitchtip.com vs django-vtasks.glitchtip.com), so vpg.glitchtip.com is an equally reasonable choice; easy to change.

The benchmark exists because the README's per-operation claims (jsonb reads, pipelined inserts, COPY batches, buffer retention) were not reproducible from this repo — the equivalent harnesses live in glitchtip-backend and are coupled to GlitchTip's models. benchmarks/driver/ab.py is a self-contained version using the repo's own bench_row.

Benchmark design

Each workload maps onto one driver feature, so a regression points at a code path rather than at "the benchmark got slower". One subprocess per cell, since RSS is only meaningful for a process that ran exactly one driver and GT_PG_BUF_CAP_BYTES is parsed once per process.

The harness verifies before it times — model instances against a psycopg-read reference, COPY against the server's own row count, ingest against the round-tripped byte count. A fast path silently returning junk would otherwise post a spectacular result.

Three corrections made during development, all worth knowing if you touch it:

  • ingest_rss is bidirectional. A write-only version reported psycopg as perfectly flat — true and misleading, since libpq flushes its send path incrementally and ratchets on receive, while tokio-postgres does the reverse.
  • The two pools spell "never recycle" differently: vpg's pool_max_lifetime=0 disables recycling; psycopg_pool computes expire_at = now + max_lifetime, so 0 expires immediately. Passing 0 to both looked symmetric and measured opposite things.
  • Pool size dominates the retention effect. A 4-connection pool hides it almost entirely; the default is 20.

Results

django-vpg 0.3.1 from PyPI, AMD desktop (6c/12t, 30 GiB), PostgreSQL 18.4, netem at 2.23 ms measured RTT. Raw JSON committed under benchmarks/results/*_031.json.

  • ASGI: drop-in +13–18% rps at −10–16% CPU/req across all 9 cells. The write regression documented for 0.2.0 is not present. Async +52–93% rps at −33–47% CPU.
  • Peak RSS under heavy-tailed ingest: 66 MB vs psycopg's 379 MB, without recycling. The cap-disabled control hits 825 MB — the vendored patch is what makes the memory profile defensible, not a bonus optimization.
  • batch: a 1.5× gap at 0 ms becomes 5.1× at 2.23 ms. Invisible without injected latency.

Not papered over: copy.write_row is slower than psycopg's (only write_rows wins), and the sync path still uses more RAM on large payloads (+27% at 512 KB, +40% at 2 MB) — better than 0.2.0's +47%/+104%, but still a regression, so the "use async for the RAM benefit" guidance stands. Both are documented in docs/limitations.md.

Caveat on the version story

These are presented as a fresh self-consistent run, not a 0.2.0 → 0.3.1 A/B. The psycopg control — which contains no vpg code — also moved ~+7.5% between my rig and the July one, so conditions differ. A real version comparison needs both versions on one rig in one sitting.

Testing

  • Harness smoke-tested locally and on the bench box; all 19 cells pass with correctness assertions active.
  • Full suite run end to end (22 min): driver A/B at 0 ms and 2.23 ms, ASGI suite, RAM footgun at 512 KB and 2 MB.
  • ruff format --check . and ruff check . clean.
  • mkdocs build --strict clean with anchor validation on.

Not run: the Rust gates (scripts/check.sh cargo portion) — no Rust changed.

AI disclosure: Claude Code (Opus 5) wrote the docs pages, the benchmark harness, and this description, and ran the benchmark suite. Reviewed by me before merge.

🤖 Generated with Claude Code

Edited by David Burke

Merge request reports

Loading
Loading