docs: convert S22 spec and plan into storage-accounting documentation
What this does
S22 is implemented. All 21 step MRs of its plan are merged.
The spec and the plan recorded design intent that the code now carries, so this converts them the way commit dddc6cc7b converted S06's:
docs/dev/storage-accounting.mdgrows from the call-site guide it was into the full reference.docs/specs/S22-storage-accounting.mdbecomes a landing stub, so unanchored inbound links keep working.docs/plans/2026-08-04-s22-storage-accounting.mdis deleted.
The new document covers the accounting layers, the counter model and where each delta fires, the namespace_statistics and blob_storage_blobs_by_namespace schema, the emit API and its caller contracts, the buffered drain, the reconciliation pass and its candidate selection, configuration, observability, error cases, security considerations, and the known gaps.
The file's existing call-site content is kept rather than restated.
What the conversion corrected
Every claim is checked against main instead of copied from the spec.
Where the two disagreed, the document records what the code does:
| Spec or plan said | main does |
|---|---|
Shadow table columns namespace_id, sha256, size |
Physical order is namespace_id, size, sha256, and the jet model follows it |
The size_bytes recompute is one SELECT DISTINCT shape |
Only the two Maven walks use DISTINCT; the four container and npm walks deduplicate with a UNION of two arms |
artifacts_count counts version rows |
The two container branches count manifest rows |
A nine-cell (format, kind) matrix |
Twelve cells, eight positive hits, six distinct walks — docker and oci share both container walks |
components_count sums six per-table counts |
One statement with six scalar subqueries |
Only reconciliation_backlog is lease-gated |
namespaces_missing_statistics is too; both drop together |
A 1000-row cap on drain_batch_size |
The cap is on reconciliation_batch_size; drain_batch_size defaults to 20000 and has no ceiling |
An UPDATE from the shadow's MutableColumns puts size in the WHERE |
MutableColumns is empty, so it renders with no SET clause and PostgreSQL rejects it |
MountBlob mirrors CompleteUpload |
It emits no namespace-scoped delta, because a mount reuses a blob the namespace already holds |
The namespace_statistics migration needs its trigger and seed in one transaction |
The single file is an observability preference. Only the shadow table's seed needs the atomicity |
The composition-root seam is periodicRegistration |
Renamed to jobRegistration, beside a new workerRegistration |
Two decrement sites the design names have no implementation on main, and the document says so in a form that stays true after they land: the lifecycle purger emits no footprint decrements, and nothing reclaims a blob.
Both counters come down only at reconciliation until then.
Two interim npm writers survive with no caller.
Diff size
Past the 500-line threshold, so here is the split. It is one document conversion, and splitting it leaves either a spec pointing at a document that does not exist or a document duplicating a live spec.
| Group | Lines |
|---|---|
docs/dev/storage-accounting.md |
+1102 / -22 |
docs/specs/S22-storage-accounting.md (spec to stub) |
+24 / -1264 |
docs/plans/2026-08-04-s22-storage-accounting.md (deleted) |
-1505 |
| Index, inbound-link, and citation updates, 13 files | +38 / -35 |
Overlap with open merge requests
Four open MRs edit docs/dev/storage-accounting.md, so whichever lands second needs a rebase.
Flagging it here because no pipeline reports it:
- !1895 (merged) rewrites one paragraph this MR turned into a table, and appends a section after text this MR keeps.
- !1912 (merged) appends a section after the last code block, which this MR also keeps last.
- !1916 rewrites text inside two sections this MR rewrote.
- !1997 rewrites the npm first-attach paragraph, which this MR also rewrote.
!1959 edits one row of docs/plans/2026-08-11-s20a-lifecycle-closed-beta.md that this MR also touches.
!1512 edits docs/specs/S17-rest-management-api.md around 22 lines above the bullet this MR rewrote.
!1953 and !1916 edit docs/specs/S20-a-lifecycle-closed-beta.md, whose one line this MR touches.
The appended sections should apply. The in-place rewrites will conflict.
Testing
Documentation only. No code changes.
markdownlint-cli2, lychee --include-fragments, and vale pass on every changed file.
Every anchor another document links into the new file resolves, and every file path, symbol, migration name, table, index, trigger, Redis key, job kind, metric, and configuration field the document names exists in the tree.
A second pass re-checked the finished document against the tree and found twelve claims that did not hold. The second commit fixes all twelve and restores five design facts the first pass dropped. Both are listed in that commit's message. The largest were:
- The management API's emit series carries
result="ok"; the three format dispatchers do not. - The bulk workers emit inline on a worker goroutine, which the emit-site table's preamble denied.
- The marshaled payload is 64,660 bytes and 36 over at 862 scopes. The
873/39pair is the model's arithmetic, which the code says to supersede. - The shadow table's seed holds
SHARE ROW EXCLUSIVEonblob_storage_blobsand all 64 partitions until it commits, so blob writes are unavailable for its whole duration. That is a deploy-timing constraint, and it was missing.
What review changed
Twelve review threads were worked after the first two commits. Eleven of them moved text, in ten commits; the twelfth was a clean-review note with nothing to change.
## Accounting layersnames the three artifact-level write mechanisms separately again, with their call sites. The flattened sentence contradicted S17's Freshness block and left this document's own opening pointing at a section that recorded no mechanism.- The tombstone-count paragraph no longer says a lost
Δartifactsdecrement is lost for good. Everyartifacts_countwalk carries the soft-delete predicate on each level that has the column, so the recompute corrects it. - The reconciliation liveness alert names
jobs_processed_total{kind="trigger:reconciliation", status="ok"}and states that a rule written againstsuccessmatches no series. The foundation-metrics bullet gainsoktoo. - The drain cost model attributes its 80 chunks to both families' ticks, and carries the derivation from the configuration table.
- The reconciliation guard's Redis-lease rejection is scoped to its critical section, so it no longer reads as a rule about leases beside the collectors that run on one.
- The error table bounds the no-baseline drop at one
reconciliation_orphan_sweep_interval, and gains a row for a key leaving Redis whole. - S22's retired criterion numerals are answered: the three cites this MR rewrote name the emit contract instead, and the stub answers every citation elsewhere in one sentence. The management-API emit table regains the one-site invariant the spec's no-retrofit warning carried.
docs/specs/S17-rest-management-api.mdanddocs/plans/2026-08-10-maven-version-size-accounting.mdno longer quote text their link targets do not contain.
Two residual items are recorded outside this MR rather than in it. #583 carries the S17 Follow-ups bullet's remaining scope question, and #515 carries the S22 criterion citations in docs/specs/S20-a-lifecycle-closed-beta.md and the two npm emit test files.
Guardrails
- No specs, plans, ADRs, or e2e catalogs change behavior here, so no conformance run and no catalog revision applies.
- The three Go comments that name the spec path are left alone: the stub answers them, and editing them would pull three pre-existing over-cap comment blocks into the comment-caps hook.
Related to #515