chore(datastore): recompute repository artifacts_count from source (S22 plan: 11/21)
What this step delivers
RepositoryReconcileStore, in internal/datastore/reconcile_repository.go, derives a repository's artifacts_count from the rows that back it.
It writes that value back.
It also enumerates a namespace's repositories for the reconciliation walk Step 14 adds.
RecomputeArtifactsCountdispatches on(format, kind)through a map, not a switch. A totality test ranges every format against every kind, and it fails the suite on a pair nobody decided. Eight pairs resolve a counted table, across six walk builders. The four virtual pairs map to nil and answer zero with no round trip. Each virtual cell states its own reason, and the reasons split two ways. The two container-family cells name the double count that a walk over the rows they proxy folds into this column. The Maven and npm cells name the inflation that a dispatch onformatalone produces.- The soft-delete predicate lands on every level of a join chain that carries
soft_deleted_at, not on the counted table alone. See "Predicate level" below. WriteBackCounterssetsartifacts_countandlast_reconciled_aton one row, and nothing else.ListForReconcilepages one namespace's repositories onname, and it carries no soft-delete predicate of its own. A tombstoned repository still needs its counters reconciled. The page projects the persistedartifacts_countbeside(id, name, format, kind). See "Projected counter" below. The cursor isnamebecauseunique_repositories_namespace_id_and_nameis non-partial and keys(namespace_id, name)in the walk's own order. The only(namespace_id, id)-leading index is partial onkind = 2 AND soft_deleted_at IS NULL, so it drops the tombstoned rows this walk must reach. A limit at or below zero is rejected before the fetch, and so is a limit abovemaxRepositoryReconcilePageSize, which stands at 1,000.
Every table on every chain is partitioned by HASH(namespace_id).
Each walk therefore carries its own namespace_id equality, and each join matches on (id, namespace_id).
This step adds no migration, no configuration key, and no route. It has no production caller yet: Step 12 extends the dispatch, and Step 14 walks the enumeration.
Corrections to the spec and the plan
Work on this step falsified several claims in both documents. Each correction lands where the claim is, rather than being worked around in the code.
container_remote_manifests exists on main
The spec said twice that this table does not exist, and the plan said it twice more.
The table is on main, created by internal/datastore/migrations/sql/20260807130000_create_container_remote_manifests.sql.
The merge request that the spec assigned this coverage to has merged without it.
The answer is to cover container_remote_manifests.soft_deleted_at here.
The (format, kind) matrix grows from six positive cells across five tables to eight cells across six tables.
The spec and the plan now record that the table exists, and the coverage assignment points at this step.
container_images.soft_deleted_at is on main
The plan's Depends on: line for Step 11 said this column "is not on main today", and expected it from #313.
The spec carried the same expectation in three more places.
The column is on main, added by internal/datastore/migrations/sql/20260812150000_add_container_images_soft_deleted_at.sql.
This step therefore ships no migration of its own, and #313 is removed from that line.
All four places now record the column as present.
The Maven delete surface
The spec said Maven has no artifact delete path in the tree today.
The management-API package delete is served, and it stamps maven_packages.soft_deleted_at.
The spec now splits the claim by column and by reachability.
maven_packages.soft_deleted_at has one writer a request reaches on main: mavenPackageDelete calls mavenPackageDeleterAdapter, which calls MavenPackageStore.SoftDeleteMavenPackage.
maven_versions.soft_deleted_at has none, because both Maven version-delete arms answer 501 and SoftDeleteMavenVersion has no production caller.
MavenBulkMarkers is the near miss, and it is wired: MarkMavenPackage and MarkMavenVersion both run inside the registered bulk worker.
No production site constructs BulkDeleteMavenArgs, so the worker never runs, and that one fact disqualifies both markers.
ADR-004's repository cap, and issue #472 (closed)'s open half
The spec derived its per-task cost model and its concurrency sizing from ADR-004's repository cap. That cap counts live repositories per namespace per artifact type, and an operator can configure it per instance. The walk also reaches the namespace's tombstoned-but-unpurged repositories, which the cap does not count. Both places that rest on the cap now say which population it bounds.
The spec also recorded #472 (closed) decision 2 as closed in one place and as unsettled in another. The two places now agree, and they name the half of that decision that is still open.
Decisions recorded here
Predicate level: every level that carries the column
The spec put the soft-delete predicate on the counted row alone.
A parent tombstone hides its children and writes no child row.
Under the old rule, the Maven and npm walks counted rows that no read path resolves.
Every reconciliation pass then re-inflated artifacts_count for the length of a purge window.
The predicate now goes on every level that carries the column.
Six predicates sit at a parent level.
Two of the six are container-family: the parent container_images under the hosted count, and the parent container_remote_images under the remote one.
The other four are the Maven and npm parent-package predicates:
| Counted table | Parent predicate |
|---|---|
maven_versions |
maven_packages.soft_deleted_at |
maven_remote_versions |
maven_remote_packages.soft_deleted_at |
npm_versions |
npm_packages.soft_deleted_at |
npm_remote_versions |
npm_remote_packages.soft_deleted_at |
npm hosted is in that list on purpose.
CascadeSoftDeleteNpmPackage stamps each version today, and the two bare deleters mark a package only when no active version is left.
That cascade is a property of today's callers, not of the schema, so the count follows the column.
The spec's enumeration is what gave way here. Its tombstone-decrement criterion is unchanged, and the enumeration now agrees with it.
Each of the four parent-package predicates was proven by mutation.
With the predicate deleted, a named subtest fails.
..._ContainerRemoteTombstoneRoutes does the same for container_remote_images.
container_images needs no separated subtest.
It is the hosted container branch's only predicate, so the two hosted matrix cells attribute it on their own.
The container_manifests marker
The spec said that container_manifests "does not gain" a soft_deleted_at column.
That sentence contradicts ADR-007, whose container_manifests block specifies timestamptz soft_deleted_at "nullable" and whose index on that table reads WHERE soft_deleted_at IS NULL.
The sentence is inherited from !1350 (merged), not introduced here.
The sentence now says that the closed beta adds none.
It names ADR-007 as where the marker arrives.
S20-A places that arrival at GA in two sentences.
S17 places it at GA at docs/specs/S17-rest-management-api.md:1457.
S17's per-table marker-decision table, at lines 239 and 241, is narrower: it puts the marker's arrival at S20's retention window.
The GA claim therefore reads off line 1457, not off that table.
It points at #472 (closed) decision 2 for the release that carries the column.
This MR edits no file under docs/adr/, and it opens no handbook merge request.
The Go walk counts the schema as it stands.
It reads the absence of that column off internal/datastore/migrations/structure.sql rather than citing ADR-007 for it.
The statement deadline for the reconciliation task
The reconciliation task's per-statement deadline is recorded as an obligation in the plan's Step 14 entry.
The entry states what supplies no deadline today.
The store's methods state none in their contract, and the service sets no statement_timeout.
Each count also draws on the pool that serves requests.
This MR carries no Go change for it, no doc comment on the store, and no configuration knob. Step 14 owns the concurrency semaphore that bounds how many passes run at once, so the deadline value is owed to Step 14.
Autovacuum and HOT on repositories
Reconciliation gives repositories one UPDATE per row per pass, whether the counter drifted or not.
The plan already poses the autovacuum question for namespace_statistics.
The same question for repositories is now recorded beside it, in the plan's Step 13 entry.
The write-back statement also carries a comment stating the HOT property itself.
last_reconciled_at sits in no index on that table, so a pass writing an unchanged counter changes no indexed value and stays HOT-eligible.
artifacts_count is covered by index_repositories_on_namespace_id_and_artifacts_count_id, so a pass that moves the counter forfeits HOT.
Projected counter: artifacts_count on the enumeration page
ListForReconcile projects the persisted artifacts_count in this MR.
Step 14 compares each recomputed counter against the persisted one before it overwrites the column.
The projection keeps that comparison off a second read per repository.
The page already takes a heap access for id, which the index does not carry, so the extra column costs nothing at the database.
A positive per-row assertion pins the projected value against the seeded one.
Without it, a new field rides the whole-value paging comparison as a zero on both sides.
The plan's Step 11 entry records the terms, and Step 12's entry records the commitment to add size_bytes the same way.
This option carries a known consequence, and it is a consequence of the chosen option rather than a defect to correct here. The projected counter is read at page-fetch time, up to one page before that repository's write-back. The page is capped at 1,000 rows. A drain tick inside that window moves the counter. The drain's guard does not stop it, because this MR's write-back is what stamps the reconciliation timestamp. The counters stay correct. The drift histogram overstates, and that histogram is the spec's alert signal. Step 14 owes the answer on whether a page-stale value satisfies the spec's pre-override read.
Spec coverage
The table below is the test-author table, pasted as it was produced.
Two of its rows are superseded by the container_remote_manifests correction above: criterion 9's artifacts_count-excludes row, and criterion 10's repository half.
Both rows read six positive cells across five tables.
The step ships eight cells across six tables.
Three subtests exist beyond the wording those two rows carry.
..._ParentTombstoneRoutes and ..._ContainerRemoteTombstoneRoutes split each two-predicate branch into one case per route, so a failure names the level that lost its predicate.
..._ScopedToTheRequestedNamespace pins the namespace scope.
Spec: docs/specs/S22-storage-accounting.md
Step 11 owns repositories.artifacts_count only. size_bytes is Step 12 and components_count is Step 13, so rows naming either are marked as another step's rather than left blank.
Acceptance criteria
| # | Criterion | Tests |
|---|---|---|
| 1 | Repo-scoped increment reflected after the next drain tick | Steps 5-10 (buffer, drain, chunk workers). Not this step. |
| 2 | Namespace-scoped increment reflected after the next drain tick | Steps 5-10. Not this step. |
| 3 | Concurrent increments to one scope sum exactly | Steps 5-8. Not this step. |
| 4 | Re-marked scope captured with no lost delta | Steps 5, 8, 10. Not this step. |
| 5 | Chunk past drain_chunk_stale_timeout bails and re-adds |
Step 8. Not this step. |
| 6 | Chunk failing every attempt re-adds its scopes | Step 8. Not this step. |
| 7 | Redis unavailable at increment time does not fail the operation | Step 6. Not this step. |
| 8 | Reconciliation clears the buffer before the SQL scan | Step 14. Not this step. |
| 9 | Recompute soft-delete visibility, per format (artifacts_count-excludes clause) |
TestRepositoryReconcileStore_RecomputeArtifactsCount_FormatKindMatrix — six positive cells, each seeding live and tombstoned rows: npm and Maven (hosted and remote) against their own soft_deleted_at, docker and oci against the parent container_images row |
| 9 | Same criterion, size_bytes-includes clause |
Step 12. Not this step. |
| 9 | Same criterion, components_count-includes clause |
Step 13. Not this step. |
| 9 | Same criterion, does-not-re-inflate-on-a-second-pass clause | Step 14 owns the whole-pass assertion. The recompute's own idempotence is asserted here: every matrix cell calls it twice and pins the second value to the first. |
| 10 | Positive hit per version-type table and (format, kind) (repository artifacts_count half) |
..._FormatKindMatrix — container_manifests on both container-family formats, maven_versions, maven_remote_versions, npm_versions, npm_remote_versions; the two remote cells additionally pin the wrong-repository join through a sibling repository |
| 10 | Same criterion, namespace components_count half |
Step 13. Not this step. |
| 11 | Drift histogram observation before the overwrite | Step 14. Not this step. |
| 12 | Crash between HINCRBY and SADD still captured by reconciliation |
Step 14 (accountingfaults). Not this step. |
| 13 | Hash TTL refreshed on every write | Step 5. Not this step. |
| 14 | namespace_statistics, shadow, and repositories.last_reconciled_at migrations |
Steps 1, 2a, 2b. Step 1 is merged; this step reads the column it added. |
| 15 | Shadow-table triggers keep the shadow consistent | Step 2b. Not this step. |
| 16 | Every namespace has a zeroed namespace_statistics row |
Step 1, merged. Not this step. |
| 17 | npm publish and unpublish emit sites | Step 17. Not this step. |
| 18 | OCI increments at the actual sites | Step 18. Not this step. |
| 19 | OCI decrements at the delete handler | Step 18. Not this step. |
| 20 | Maven post-commit increments, stub retired | Step 19. Not this step. |
| 21 | Repository cascade hard-delete (gated on #464 (closed)) | Owned by the S20-a plan, verified in that work's MR. Not this step. |
| 22 | One asynq task per namespace candidate | Step 14. Not this step. The per-namespace repository walk that task drives is covered here by TestRepositoryReconcileStore_ListForReconcile. |
| 23 | Namespace with no statistics row gets one on its first pass | Step 13. Not this step. |
| 24 | reconciliation_max_in_flight never exceeded |
Step 14. Not this step. |
| 25 | Source-first ordering at every emit site | Steps 6, 17, 18, 19. Not this step. |
| 26 | last_reconciled_at stamped only after every repository is written back |
Step 14 owns the namespace-level stamp. The repository-level half — each write-back stamps its own row and no other — is TestRepositoryReconcileStore_WriteBackCounters/overwrites_the_counter_and_stamps_only_its_own_row. |
| 27 | Staleness selection and orphan sweep | Step 15. Not this step. |
| 28 | UniqueByArgs caps outstanding tasks at one |
Step 14. Not this step. |
| 29 | reconciliation_backlog single-writer collector |
Step 16. Not this step. |
| 30 | Drain chunk's reconciliation guard skips a reconciled scope | Steps 7, 8 (accountingfaults). Not this step. |
| 31 | counter_dirty_set_size sampled once per tick |
Step 10. Not this step. |
| 32 | Config load rejects each invalid configuration | Steps 3, 3b, both merged. Not this step. |
| 33 | The six S22 metrics registered; alerts wired | Steps 8, 10, 14, 16; the alert half ships with #354. Not this step. |
| 34 | Saturation returns for re-enqueue, never sheds or blocks | Step 14. Not this step. |
| 35 | Namespace-scoped chunk with no namespace_statistics row |
Steps 7, 8. Not this step. |
| 36 | Recovery SADD failure loses no delta |
Steps 8, 10 (accountingfaults). Not this step. |
| 37 | Management-API delete emits (gated on #313) | Deferred with #313; no plan step. Not this step. |
Error cases
| # | Condition | Tests |
|---|---|---|
| E1 | Redis unavailable at increment time | Step 6. Not this step. |
| E2 | Redis unavailable at drain-trigger time | Step 10. Not this step. |
| E3 | Chunk job's Postgres UPDATE fails |
Step 8. Not this step. |
| E4 | Chunk job's :flushed DEL fails after the UPDATE |
Step 8. Not this step. |
| E5 | Chunk job exhausts all retry attempts | Step 8. Not this step. |
| E6 | Recovery SADD itself fails |
Steps 8, 10. Not this step. |
| E7 | Chunk dequeued past drain_chunk_stale_timeout |
Step 8. Not this step. |
| E8 | Worker dies mid-chunk after merging into :flushed |
Step 8. Not this step. |
| E9 | Two chunks run the same scope concurrently | Steps 8, 10. Not this step. |
| E10 | Trigger's EnqueueTx fails while the process is alive |
Step 10. Not this step. |
| E11 | Crash between the trigger's SPOP and its EnqueueTx |
Step 10. Not this step. |
| E12 | Assigned repository or namespace row hard-deleted before its chunk drains | Repository half at this layer: TestRepositoryReconcileStore_WriteBackCounters/is_a_no-op_for_a_repository_row_that_is_gone. The drain chunk's own half is Step 7; the namespace half is Step 13. |
| E13 | Namespace-scoped chunk drains a namespace with no statistics row | Steps 7, 8. Not this step. |
| E14 | Crash between a scope's HINCRBY and its SADD |
Step 14. Not this step. |
| E15 | Crash between reconciliation's clear and its SET |
Step 14. Not this step. |
| E16 | Reconciliation scan races a concurrent increment | Step 14. Not this step. |
| E17 | A drain chunk and a reconciliation process one scope concurrently | Steps 7, 8. Not this step. |
| E18 | Reconciliation finds a discrepancy | Step 14. Not this step. |
| E19 | Namespace has no namespace_statistics row when its task runs |
Step 13. Not this step. |
| E20 | Reconciliation task fails before its final UPSERT | Step 14. Not this step. |
| E21 | A namespace can never be reconciled | Steps 14, 15. Not this step. |
Security considerations
| # | Concern | Tests |
|---|---|---|
| S1 | Redis keys carry only internal UUIDs, no user-controlled text | Step 5 owns the key grammar. This step writes no Redis key. |
| S2 | Counter values are non-secret but feed billing, so an inflating or deflating bug has financial impact | Partly here. The recompute is the correction path the criterion rests on, and the matrix pins both failure directions for artifacts_count: a dropped soft-delete predicate inflates, and a (format, kind) fall-through deflates. The billing-input columns themselves are Step 13's, and the drift metrics are Step 14's. |
| S3 | No new credential surface; Redis and Postgres reuse existing clients | This step adds no client. NewRepositoryReconcileStore takes the datastore pool the other stores take. |
e2e scenario catalog
No change to docs/testing/.
Guardrail 12 scopes that catalog to feat and fix work, and the plan gives Step 11 Type: chore.
The step also changes no runtime behavior: no endpoint, no route, no middleware, no configuration key, no storage path, and no migration.
No configuration-reference or Bruno pairing is owed either.
The diff touches none of internal/config/**, proto/artifactregistry/config/**, config.example.yaml, and api/openapi/**.
Diff size (guardrail 18)
The plan's dependency-and-estimate table carries ~400 Go LOC for this step.
The branch delivers 2,648 added Go lines, plus documentation edits to the plan and to the spec.
Measured at 58feb492, against merge base a5cf9da6, which git merge-base origin/main HEAD gives.
Added counts come from git diff --numstat a5cf9da6 HEAD.
Reviewable counts come from grep -cvE '^[[:space:]]*(//.*)?$' over each file, which drops blank lines and whole-line // comments.
| File group | Added | Reviewable |
|---|---|---|
Production Go — internal/datastore/reconcile_repository.go |
+776 | 336 |
Production Go — internal/datastore/query_names.go |
+8 | 8 |
Integration tests — internal/datastore/reconcile_repository_integration_test.go |
+1,243 | 793 |
Query-plan tests — internal/datastore/reconcile_repository_explain_integration_test.go |
+319 | 144 |
Untagged unit tests — internal/datastore/reconcile_repository_test.go |
+302 | 229 |
Documentation — docs/plans/2026-08-04-s22-storage-accounting.md |
+86 / −32 | — |
Documentation — docs/specs/S22-storage-accounting.md |
+45 / −26 | — |
Production is +784 added and 344 reviewable.
Tests are +1,864 added and 1,166 reviewable.
Documentation is +131 / −58.
query_names.go is the second production file, and it was already on main, so its reviewable count is its eight added lines rather than a whole-file count.
Every other row's file is new on this branch, so the whole-file count is the added count.
The plan's Step 11 paragraph is measured at this same head against this same base, so the two now read the same numbers.
The plan carries this interrogation in its "Seam interrogation outcomes" section, and it keeps the step bundled. A split does not help here, for four reasons:
- The estimate was wrong about the test surface, not the production one. Six recompute walks, the keyset enumeration and the write-back come to 306 reviewable production lines, which is inside the ceiling on its own. A split of the production file therefore removes nothing a reviewer has to hold at once.
- The rest is the
(format, kind)matrix and the fixtures it needs. Criteria 9 and 10 ask for the cells one by one, so a dropped cell drops a positive hit that a criterion names. A split that defers cells leavesmainwith those criteria uncovered until the second MR lands. - The step exposes no foundational type below itself to extract.
Step 12 extends
RepositoryReconcileTargetand the store, and Step 14 walks the enumeration. To hoist either one inverts those edges, so the extracted MR then depends on the step that consumes it. - The one internal boundary that looks extractable is the enumeration.
A split there ships a walk with nothing to recompute, or a recompute with no way to reach a repository.
Neither half is reviewable against this step's
Acceptance, which asserts the two together.
The plan asks for two review passes instead of a split: the dispatch and its six walks first, then the enumeration and the write-back.
The documentation edits are of two kinds, and neither is production surface.
The first kind is the stale-main corrections above.
The second kind is the review-decision records this MR is the MR of record for.
Each of those records lands in the entry of the step its decision belongs to, rather than in Step 11's.
Tagged lint (guardrail 7)
CI lint compiles no //go:build integration file, so guardrail 7 asks for a build-tagged run.
This is the command, and the result measured at ac2fa0cf:
golangci-lint run --build-tags=integration --max-same-issues=0 --max-issues-per-linter=0 ./internal/datastore/It exits 1 with 3,570 findings in the package.
An earlier run of the same command reported 3,489, at ac2fa0cf against merge base 45b19f60.
The branch has since rebased onto a5cf9da6, 83 commits further on, so the two totals are measured over different package contents and their difference is not this branch's.
Read the result as a composition rather than as a bare delta.
Forty of the 3,570 findings sit in this branch's own reconcile_repository* files, and all 40 are contextcheck.
That is one pre-existing package-wide class: a shared test helper called without the context parameter.
The same class accounts for 3,326 further findings, in suites this branch does not touch.
No other linter reports on any of this branch's files, query_names.go included.
lll reports 6 findings in the package, and none of the 6 is in a file this branch touches, which is what the wrapped instrumentExec call is measured against.
Merge order and dependencies
- Step 1 merged as !1422 (merged).
It added
namespace_statisticsandrepositories.last_reconciled_at. This step reads that column and stamps it. - Both schema dependencies are on
main:20260807130000_create_container_remote_manifests.sqland20260812150000_add_container_images_soft_deleted_at.sql. This step ships no migration. - Step 12 (
size_bytes) and Step 14 (the reconciliation task) depend on this step. Step 12 extends the dispatch in place, and Step 14 walksListForReconcile. - This step widens no predicate ahead of the code that serves it. Neither the recompute nor the enumeration has a production caller, and Steps 12 and 14 add the first ones.
Seven other merge requests touch what this one is measured or shaped against. One has merged: !1611 (merged). Each note on the other six reads correctly whether or not that merge request has merged.
- !1611 (merged) has merged, and this branch has rebased onto it.
It added
instrumentQueryandinstrumentExectointernal/datastore, and itsTestEveryStatementIsInstrumentedfails any function in that package that runs a statement bare. It puts the same rule indocs/dev/database-query-patterns.md. All three of this branch's statements now go through those helpers:ListForReconcileandRecomputeArtifactsCountthroughinstrumentQuery, andWriteBackCountersthroughinstrumentExec. Eight names inquery_names.gocarry them, each appended to its target table's group: the enumeration page, the six recompute walks, and the write-back.ListForReconcile's first page and its cursor page are one statement with one optional predicate, so they share a name; the catalog's one-name-per-statement rule counts the round trips a method issues, not predicate branches.RecomputeArtifactsCounthas one call site for six statements, so the name travels with the builder inartifactsCountWalkrather than at the call site:TestQueryNames_EachUsedExactlyOnceadmits a name at exactly one site, anddockerandocieach reach the two container walks through a key of their own. No statement builder's body or signature changed, so the plans in the Database Review Evidence notes still describe the SQL this store issues. Nothing here is waiting on anything now. - !1514 (merged) adds a
soft_deleted_at IS NULLfilter toRepositoryStore.FindByID. That filter is why the rejected option for the projection decision was rejected. The rejected option read the persisted counter throughFindByIDat drift time. If !1514 (merged) merges first, that read raises a not-found error for exactly the tombstoned repositories this walk exists to reach. - !1513 (merged) lands
RepositoryStore.SoftDelete, the writer that stampsrepositories.soft_deleted_at. The closed-beta argument in the spec rests on that writer. The other half of that premise needs no merge request:RepositoryStore.Listalready filterssoft_deleted_at IS NULLonmain, which is why the enumeration cannot reuse it. Until !1513 (merged) merges, the walk's tombstone cases are reachable only from a test seed. Once it merges, a production path writes them. - !1510 (merged) and !1551 (merged) each rewrite spec text this MR also edits. !1510 (merged) edits near the shadow-table section, and !1551 (merged) rewrites the accounting-layers section. Neither one edits a line this branch changes, so the second of the two to merge carries a textual rebase rather than a content conflict.
- !209 resplits the LOC budget into a production ceiling and a separate test budget. That is the budget the "Diff size" section above measures against. !1559 (merged) forbids a step MR from editing the plan file at all, Status row included. This branch fills Status row 11, which guardrail 4 requires while !1559 (merged) is open. If !209 merges, the "Diff size" section is read against its split budget instead of one ceiling. If !1559 (merged) merges, that Status edit becomes the shape its new rule forbids.
Documentation pairing
The trigger did not fire.
datastore.RepositoryReconcileStore gets no page under docs/dev/, for two reasons:
- The merged plan ran the doc-pairing question for each step and answered "none" for Step 11.
It names
docs/dev/pages for Steps 6, 10, and 14, so the answer is a per-step one. To re-open a pairing that the plan MR settled is outside this MR's scope. - The shape of an
internal/datastorestore is documented by symbol already, indocs/dev/database-query-patterns.mdunder "Datastore package layout". No datastore store has a page of its own.
This step's deviations from that section are recorded in the plan.
The walk takes validKeysetText from internal/datastore/keyset.go and leaves the rest of that file alone.
probeLimit and splitProbeRow serve a hasMore contract this store does not have.
RepositoryReconcileTarget also scans its five columns straight into an alias-tagged struct of its own.
The guide's "Projecting a column subset" section asks for a scan into the generated model type and a conversion.
The plan records that deviation, and it records why the safety the conversion buys holds by construction here.
Database Review Evidence
Query mode ran and posted its plans in note 3695801747.
That note supersedes note 3692659003, which measured the enumeration before it projected artifacts_count.
The earlier note stays in place, unedited.
The new note measured at the commit docs(plans): re-measure Step 11's landed LOC figures, which the rebase onto a5cf9da6 renumbered from a12f745a to 077ab7f8.
Four commits follow it, and none of them changes a statement: two edit Markdown, one adds a comment paragraph to the query-plan suite, and 26ac5a73 routes the three call sites through instrumentQuery and instrumentExec.
That last one leaves every statement builder's body and signature byte-identical, which is what keeps the posted plans a description of the SQL this store issues.
The head is named by commit subject rather than by a SHA alone, because a rebase discards the SHA and leaves the sentence pointing at nothing.
Migration mode did not run, and the note records why rather than leaving it implied.
The diff adds no migration.
Both columns the new code reads are already in internal/datastore/migrations/structure.sql at the merge base, and so are both indexes the statements ride.
Plans come from EXPLAIN (ANALYZE, BUFFERS) on an ephemeral PostgreSQL 17.10 container, with the branch's migrations applied to 20260814150911.
The seed data is synthesized, and every transaction rolls back.
The timings below hold at moderate cardinality.
They do not capture production-scale effects.
Nine statements come out of the store's three methods: two enumeration pages, the six recompute walks, and the write-back. The four virtual pairs issue no statement, so no plan exists for them. No statement carries a partition fan-out.
The fifth projected column is free
This is the claim the re-run exists to check, and the measurement holds it.
unique_repositories_namespace_id_and_name keys (namespace_id, name) and nothing else, and the projection also needs id, format, and kind.
The page therefore took one heap access per row before this change, and artifacts_count rides that same access.
One transaction planned both projections over one 5,000-row seed, at the 1,000-row page cap.
Four things are identical across the two: the plan node, the index, the Index Cond, and the total cost of 0.28..122.16.
The buffer count is identical too, at shared hit=27 with no reads.
Only the row width moves, from 44 bytes to 52, which is the one added bigint.
Three interleaved rounds gave overlapping times, so the added column costs nothing this method can measure.
The count walks and the write-back
All six recompute walks take one plan shape, and each prunes all three tables on its chain to one partition of 64. On the base seed the planner reads the estimate off the seed exactly, and execution stays between 0.669 ms and 1.325 ms. The counted partition takes a Seq Scan there because every row in it belongs to the counted repository, so no index can read less. A second seed puts the same 5,000 counted rows beside 45,000 rows of another repository in one partition. The planner then takes a Bitmap Index Scan. The repository's own rows therefore bound the walk rather than the partition. That second seed is also the one place an estimate diverges, at 25,000 rows planned against 5,000 actual.
Predicate placement matches the source comments.
The hosted container walk shows its soft-delete predicate on the parent container_images partition alone, and each of the other five shows both of its own.
The write-back reaches one partition of 64 in every measurement, and both key columns always reach the plan as equalities.
Which access method carries them changes with the row count of the addressed partition.
Under the default enable_seqscan, a small partition takes a Seq Scan with both keys in the Filter, and from 1,000 rows up both keys go inside one Index Cond.
Note 3695859700 corrects one claim of note 3695801747 here.
That note called the small partition and the enable_seqscan pin two reasons for the Filter shape.
The pin is the regime the query-plan suite runs in, not a second cause.
Inside that regime one row gives the Filter shape, and fifty rows give both keys in one Index Cond.
The suite's comment on that plan is therefore accurate as written, and its clause-agnostic assertion holds across every row count measured.
Controls
A variant that drops the maven_packages namespace equality from the WHERE clause alone plans identically.
Both composite joins still carry that equality, so the planner derives the constant.
A variant that drops it from the WHERE clause and from both join conditions fans out over all 64 partitions of that table.
Both variants are controls written for this measurement, and neither is a statement this branch contains.
Neither the query-mode note nor its correction leaves an anomaly open, and neither raises anything that waits on a decision.
The page-size bound the earlier note asked for is on the branch, enforced at maxRepositoryReconcilePageSize.
A query-plan suite also pins three of these properties in-tree, rather than leaving them in a note. The first is the enumeration's index with its folded keyset bound and no sort node. The second is one-partition pruning on every table each of the six recompute walks joins. The third is the write-back's prune to one partition, with the single row it addresses pinned.
The plan requires the line below on every one of its twenty-one step merge requests, feat and chore alike.
Related to #515
This is a bot message