Remote eviction leaves repository counters stale: widen the delta composers to the npm, Maven and Container arms

The Maven bulk remote eviction arm emits no storage-accounting counter deltas. The delta composers cover the hosted arm only, so a remote eviction leaves two namespace and repository counters stale until reconciliation or the reap corrects them.

Today the whole record of this gap is one comment, remoteOps() in internal/managementapi/bulk_maven_worker.go. Nothing under docs/ and no plan records it, so a reader who does not open that function has no way to learn the obligation exists. This issue is that record.

Which counters go stale, and which do not

Counter On a remote eviction Why
repositories.artifacts_count Stale, too high evictMavenRemotePackageStmt and evictMavenRemoteVersionStmt are UPDATE ... SET soft_deleted_at, and recomputeMavenRemoteVersionsStmt takes soft_deleted_at IS NULL at both levels, so the recompute excludes rows the emit never decremented.
repositories.size_bytes Stale, too high deleteMavenRemoteFileStmt is a DELETE, and recomputeMavenRemoteFilesSizeStmt sums distinct blob sizes over maven_remote_files.
namespace_statistics.components_count Unmoved recomputeNamespaceComponentsCountStmt is six bare row counts with no predicate at any level, and its own doc says the column settles at hard delete. A tombstone does not move it.

docs/specs/S22-storage-accounting.md line 1002 states the same rule for these tables: the repository's artifacts_count counts only the live rows while the namespace's components_count counts both.

Why this needs its own record

The remoteOps() comment does not only describe the gap, it instructs the next author to widen the composers. An author who acts on it as written emits a components_count delta at the mark. That delta reads under the physical row count for the whole purge window, is overwritten by reconciliation, and is then emitted a second time by the reap.

The correct widening emits artifacts_count at a package or version eviction and size_bytes at a file eviction, and leaves components_count alone until the reap — the same shape deleteCounterTargetTombstone already records for the hosted arm.

Scope

All three remote arms. The delta composers widen to the remote path for npm, Maven and Container.

The three arms do not owe the same deltas, and that difference is the reason the Container arm was filed apart at first.

Arm What it writes Deltas owed
npm, Maven UPDATE ... SET soft_deleted_at on package and version rows, plus a hard DELETE of the file row Δartifacts at the mark, Δsize at the file delete
Container UPDATE ... SET soft_deleted_at only Δartifacts

Each arm reports the rows a mark hid, and the two file-bearing arms also report the bytes a file delete freed.

The Container arm, folded in from work item 837

ContainerRemoteEvictor.EvictContainerRemoteImage and EvictContainerRemoteManifest in internal/datastore/container_remote_eviction.go set soft_deleted_at, and nothing on that path references internal/accounting.

This arm hard-deletes no file row, so three of the four counters correctly do not move:

Counter On a Container remote eviction Why
repositories.artifacts_count Stale, too high The eviction sets soft_deleted_at, and the container remote count walk predicates on both container_remote_images and container_remote_manifests being live, so the recompute drops rows the emit never decremented
repositories.size_bytes Unmoved, and right not to move The eviction hard-deletes no file row, and the remote size walk carries no soft-delete predicate
namespace_statistics.components_count Unmoved, and right not to move The namespace count is physical at every level, so a tombstone does not move it
namespace_statistics.deduplicated_size_bytes Unmoved No blob row is touched. Reclamation is S28's

So the Container arm owes one delta, Δartifacts, where the npm and Maven arms owe two.

Merge order: this issue outlives !2003 (merged)

!2003 (merged) is open and implements the npm and Maven arms only. That merge request carries a Related to line rather than a Closes line, so it does not close this issue on merge, and that is correct.

This issue stays open after !2003 (merged) merges. What remains at that point is the Container arm alone. Do not close this issue on the strength of !2003 (merged).

  • #761 (closed) — the same family on the npm side: a delete path whose bytes never reach the size_bytes emit.
  • #834 (closed) — the fill half of the same matrix, for npm and Container. The Maven fill arm landed in !1901 (merged).
  • The remoteOps() comment's accuracy was reviewed on merge request !1754 (merged), which is where this gap was traced. Whatever that merge request settled about the comment's wording, the composer widening itself stays outstanding until an implementation lands for every arm.
Edited by Pawel Rozlach