Artifact-level accounting — Maven: wire maven_remote_versions.size_bytes on the cache fill

Context

docs/adr/007_database_schema.md defines artifact-level storage accounting in full. The two sections are #### Artifact-level storage accounting and #### Artifact-level storage accounting reconciliation. S22 excludes this layer and hands it to the format slices. Issue #441 (closed) defers the per-format work and asks for one issue per format. This is the Maven issue.

Every line coordinate in this body is measured against origin/main at 0f441dd08c97b95d824eb488f3e42a41f529d2a1, on 2026-09-01. ADR-007 is quoted from the handbook at 3df8af65859c422aba6f3c59490293204ffadfc3, which is the authority for the ADR text.

The hosted column exists now. An earlier version of this body said that no artifact-level accounting column exists at all on the hosted tables. That statement was true when the issue opened, and it is false today. maven_versions.size_bytes is on main, with its set-based backfill and its size-ordered partial index. The three migrations are internal/datastore/migrations/sql/20260813150000_add_maven_versions_size_bytes.sql, 20260813150100_backfill_maven_versions_size_bytes.sql and 20260813150200_add_maven_versions_size_bytes_index.sql. The index is index_maven_versions_on_ns_id_pkg_id_size_bytes ON maven_versions (namespace_id, maven_package_id, size_bytes DESC) WHERE soft_deleted_at IS NULL, at internal/datastore/migrations/sql/20260813150200_add_maven_versions_size_bytes_index.sql:41.

One item of the original eight is left, and it is the remote half. maven_remote_versions.size_bytes has a writer and no production caller. The Maven remote cache fill must call that writer with a version-scoped delta.

What landed, and where

The merged plan is docs/plans/2026-08-10-maven-version-size-accounting.md. Its Status table has six rows, and every row carries a merged merge request. This table maps the original eight work items onto those merge requests. These merge request numbers come from the merge commits on origin/main, not from the plan text.

Original work item Merge request
1. maven_versions.size_bytes column, set-based backfill, and size_bytes DESC partial index !1441 (merged)
2. Maintain the counter per ADR-007, deduplicated within the version !1551 (merged)
3. Emit from a post-commit site on the upload path !1551 (merged)
5. Artifact-level reconciliation for Maven !1676 (merged)
6. Total maven_files (namespace_id, maven_version_id) index for the recompute !1551 (merged)
7. Correct S10's buffered-column enumeration !1551 (merged)
8. Serialize the version size through S17 !1693 (merged)

Item 4, the remote twin, split. Its writer landed in !1664 (merged) as MavenRemoteVersionStore.AddMavenRemoteVersionSizeBytes. Its caller is what this issue now holds, and that caller is the whole of this issue's scope.

Two details of the table need saying, so a reader can check it against the tree. Item 6 landed as index_maven_files_on_ns_id_ver_id over (namespace_id, maven_version_id, blob_sha256), which is one column wider than the item asked for. Item 2's post-commit maintenance is at internal/format/maven/upload.go:887, which dispatches RecomputeVersionSize through bufferedUpdate(labelVersionSizeBytes, …) after the commit.

What exists today

Part State on origin/main at 0f441dd0
The column Exists. internal/datastore/migrations/sql/20260731130000_create_maven_remote_versions.sql:23 declares size_bytes bigint NOT NULL DEFAULT 0.
The size-ordered index index_maven_remote_versions_on_ns_id_pkg_id_size_bytes over (namespace_id, maven_remote_package_id, size_bytes DESC) WHERE soft_deleted_at IS NULL, at internal/datastore/migrations/sql/20260731130000_create_maven_remote_versions.sql:231. It sorts on zeros, which docs/specs/S14-maven-remote.md:146 records.
The writer MavenRemoteVersionStore.AddMavenRemoteVersionSizeBytes, internal/datastore/maven_remote_versions.go:106.
The caller None. git grep AddMavenRemoteVersionSizeBytes over origin/main returns doc comments, plan lines, spec lines and tests, and no production call.
The size signal at the seam remote.CacheEntry.Size, internal/remote/interfaces.go:63. committedRemoteFillDeltas already reads it.
The fill's result datastore.MavenRemoteUpsertResult, internal/datastore/maven_remote_cache.go:534.

The counter is version-scoped, and the seam is not

ADR-007 line 2174 names maven_remote_versions as one of the four tables the next sentence governs. Line 2176 is that sentence:

It is therefore maintained as a buffered counter via buffered/async writes, like repositories.size_bytes: it increments when a blob_sha256 first becomes attached to the version and decrements when the last attachment of that sha256 leaves the version (deduplicated within the version, matching the DISTINCT blob_sha256 used in reconciliation), and two paths remove it.

MavenRemoteUpsertResult declares three fields, at internal/datastore/maven_remote_cache.go:534. This quote leaves out each field's own doc comment.

type MavenRemoteUpsertResult struct {
	VersionCreated bool
	RepoHeldBlob   bool
	Displaced      *DisplacedBlob
}

The datastore must widen before any caller can be correct. There are two separate gaps. If only one of them closes, a caller still cannot compute a delta. These two gaps are what make the residual a unit of work rather than a loose end.

Gap 1: the seam produces no version-scoped membership fact. RepoHeldBlob answers whether the repository already held the digest. ADR-007 asks whether the version already held it. The two differ on a real case. One jar is cached under version 1.0 of an artifact, and then under version 2.0 of the same artifact. The second fill reads RepoHeldBlob true, so a caller that keys on it charges version 2.0 nothing. Displaced carries the mirror gap. A digest can stay referenced elsewhere in the repository and still leave this version. That case owes a version-scoped decrement and no repository-scoped one, and Displaced is nil in exactly that shape.

The SQL sets the repository scope, not only the doc comment, and that is what makes this gap structural. remoteRepositoryHoldsBlob (internal/datastore/maven_remote_cache.go:1455) fills RepoHeldBlob, and readFillMembershipBefore (:938) reaches it. Its statement mavenRemoteRepositoryHoldsBlobStmt (:1486) carries no version predicate, and it reaches files through maven_remote_packages rather than through the nullable maven_remote_version_id. The doc comment at :1430-:1432 says that shape is deliberate. A join through the version id drops every cached package-level entry from the repository walk this probe mirrors. So the fix is not a change to that statement. The seam needs a second, version-scoped fact beside it, plus a version-scoped displacement.

Gap 2: the result carries no version id. upsertCacheEntry computes versionID at internal/datastore/maven_remote_cache.go:802 and never surfaces it. MavenRemoteUpsertParams (:503) does not carry it either, so RemoteCacheStore.UpsertCacheEntry (internal/format/maven/remote_store.go:219) never sees it. Given a correct delta, a caller still has no remoteVersionID to pass to AddMavenRemoteVersionSizeBytes.

Work

One item is left. Call AddMavenRemoteVersionSizeBytes from the Maven remote cache fill with a version-scoped delta.

The item has three parts:

  1. Widen the datastore result, closing Gap 1 and Gap 2. UpsertCacheEntry must return the maven_remote_versions row id, a version-scoped first-attach fact, and a version-scoped displacement.
  2. Derive the per-version delta. committedRemoteFillDeltas (internal/format/maven/remote_fill_emit.go:126) derives the three existing deltas from the result's facts, and the new one belongs beside them.
  3. Dispatch it from the post-commit arm of UpsertCacheEntry (internal/format/maven/remote_store.go:265-:280).

The same change corrects the merged claims that say the column has no caller:

  • internal/datastore/maven_remote_cache.go:1103-:1115, the doc on upsertMavenRemoteVersion. Four of its sentences go false, and one is false already: remote.CacheEntry does carry a size field.
  • internal/datastore/maven_remote_versions.go:98-:99, the writer's own doc, which names the cache-fill path as the future first caller. The TODO(s18-buffered-counters) at :137 names the same unlanded caller.
  • docs/specs/S14-maven-remote.md:135 and :146, the column's "no emitter owns this column yet" note and the index's "sorts on zeros" note.
  • docs/dev/storage-accounting.md:2281, which says "nothing writes maven_remote_versions.size_bytes".
  • internal/managementapi/version_file_resources.go:85 and :88, which say "no code writes it" and "has no caller outside its own tests".

One nearby comment needs no correction, and it is named so that nobody over-corrects it. internal/managementapi/version_file_resources.go:92-:94 states a condition and its event: "While no emitter exists this is every remote row; once the cache fill that owns the counter starts writing it, the mapper reads the column the way versionFromMavenModel does." That sentence reads correctly on both sides of the caller's landing.

Emit route

The route is settled. The delta goes through the format layer, off the request path, after the fill's transaction commits. It never goes inside the cache-fill transaction.

Three written sources agree:

  • The writer's own doc comment, internal/datastore/maven_remote_versions.go:93-:94: "Callers apply deltas off the request path via the format layer's bufferedUpdate wrapper, never inside the cache-fill transaction".
  • ADR-007 line 2176, which maintains the column "as a buffered counter via buffered/async writes".
  • The merged plan, docs/plans/2026-08-10-maven-version-size-accounting.md:107 and :213, which specifies the same off-path contract for this writer.

The alternative route, a datastore-layer counter write after its own RunInTx returns, contradicts all three sources, and this issue does not take it.

Hazard: the fill's short-circuit hides a version-scoped delta

This hazard is pre-existing on main. This issue records it so that whichever merge request lands the caller handles it on purpose. It is not a new work item, and it does not widen this issue's scope.

internal/format/maven/remote_store.go:272 returns early when deltas.movesNothing() reports true. counterDeltas (internal/format/maven/upload_emit.go:218) carries three fields, and all three feed repository- and namespace-scoped counters: newVersions, repositorySize and namespaceSize. movesNothing() (internal/format/maven/upload_emit.go:264) compares the value against the zero value of that struct.

One case shows the effect. A fill carries a digest that is new to the version, into a version row that already exists. The repository and the namespace already hold that digest. VersionCreated reads false, RepoHeldBlob reads true, and remote.CacheEntry.Deduplicated reads true. All three deltas are then zero, so movesNothing() reports true and the dispatch never runs. The per-version counter meanwhile owes the blob's whole length. This case is the one ADR-007's dedup-within-the-version rule exists for.

Whatever route carries the per-version delta, that guard must see it.

Known gap: no version-scoped recompute for the remote table

ADR-007's reconciliation walk is scoped to a single maven_version_id or npm_version_id (line 2201), and it names neither maven_remote_files nor a remote version id. The tree matches that: internal/datastore/reconcile_repository.go holds a row-count recompute (recomputeMavenRemoteVersionsStmt, :775) and a repository byte-sum recompute (recomputeMavenRemoteFilesSizeStmt, :1151), and no version-scoped size recompute. So the maintained counter has nothing to reconcile against today. This body records the gap, and the gap is not an acceptance item for the caller.

Out of scope

  • The hosted arm. maven_versions.size_bytes, its backfill, its index, its post-commit maintenance and its reconciliation all landed with the merged plan.
  • The npm twin. npm_remote_versions.size_bytes and its tarball cache fill belong to issue #640.
  • The repository- and namespace-scoped counters the Maven remote fill already emits. RemoteCacheStore.emitCommittedFillCounters (internal/format/maven/remote_fill_emit.go:165) records them, and !1901 (merged) landed that work for issue #835 (closed).
  • maven_packages.last_downloaded_at and maven_versions.last_downloaded_at. S18 owns both. They are named here only so the boundary stays explicit.

Done when

  • A committed version-level cache fill of a digest the version did not hold raises that version's maven_remote_versions.size_bytes by the committed blob's byte count.
  • A refill of the same coordinate with byte-identical content leaves the counter unchanged.
  • A refill that repoints the coordinate to a different digest moves the counter by the new blob's size less the displaced blob's size.
  • A fill of a digest the version already holds under a different file name leaves the counter unchanged, which is ADR-007's dedup-within-the-version rule.
  • A fill of a digest new to the version, but one the repository and the namespace hold, still moves the counter and reaches the dispatch.
  • A package-level file creates no version row and moves no version's counter.
  • An upsert whose transaction rolled back moves no counter.
  • A failed counter write leaves the fill successful.
  • The same merge request corrects the merged claims listed under ## Work.
  • No GC-side emitter is required, and none is added. ADR-007 gives the removal side to the format's own delete and to the lifecycle purger. The purger drops the version row together with its counter.

Refines #441 (closed).

🤖 Triage note (automated): This issue has been reviewed and classified as type::feature.

Reasoning: This issue requests the implementation of net-new functionality — specifically, adding the maven_versions.size_bytes accounting column (which does not yet exist), wiring up a writer for maven_remote_versions.size_bytes, implementing ADR-007's artifact-level reconciliation for Maven, and serialising the version size through the API. None of these are regressions or broken behaviour; they are new capabilities being built out for the Maven format slice.

The following labels have been confirmed on this issue:

  • type::feature
  • Category:Artifact Registry
  • devops::package
  • group::package registry

If this classification doesn't look right to you, please feel free to update the labels and leave a comment explaining the correction. Thank you! 🙏

Re-scoped on 2026-09-01 from the original eight-item scope to the remaining cache-fill caller, and the run's ledger holds the pre-edit body verbatim.

This is a bot message 🤖 — /smurfit

Edited by Pawel Rozlach