Remote cache fill: a retried fill emits no namespace dedup bytes when an earlier attempt committed the blob

Summary

All three remote cache fills commit the payload blob before they write their cache rows, and the storage-accounting emit is attached to the row write. A fill that commits the blob and then fails leaves the blob_storage_blobs row behind with no emit for it. The retry's Commit reports that digest as deduplicated, so the retry emits Δdedup_size = 0 and namespace_statistics.deduplicated_size_bytes reads low until a reconciliation pass recomputes it.

repositories.size_bytes is unaffected, because it keys on the repository's own rows, which the failed attempt never wrote, so the retry charges it in full. The asymmetry between the two columns is the shape to look for.

Where it applies

One cause, three consumers. internal/remote/fetch.go commits at storage.Session.Commit and then calls CacheStore.UpsertCacheEntry, passing stored.Deduplicated through as remote.CacheEntry.Deduplicated. Each arm reads that field to decide its namespace delta:

  • internal/format/maven/remote_fill_emit.go:141
  • internal/format/npm/npmremote/remote_fill_emit.go:97
  • internal/format/oci/remote_fill_emit.go:45

None of the three remote grids in docs/dev/storage-accounting.md records the window today.

Reproduction

Observed on the container arm against a kind=remote repository filling from registry-1.docker.io, by failing the fill inside its own transaction (renaming the column its blob probe reads) and then restoring it:

attempt 1 (fill fails):    status 500
  container_remote_manifests rows for the digest: 0
  blob_storage_attachments rows for the digest:   0
  blob_storage_blobs rows for the digest:         1   <- payload survives the rollback

attempt 2 (fill succeeds): status 200
  ar:counter:repo_scoped:{<ns>:<repo>}   artifacts 1   size 1025
  ar:counter:namespace_scoped:{<ns>}     components 1  dedup_size 0   <- never credited

A reconciliation pass then reported the gap and corrected it: deduplicated_size_bytes 4224757 -> 4225782.

The injection is artificial; the window is not. Any failure inside the fill's own transaction reaches it, and so does a crash between the two commits.

Documenting it belongs in one place

docs/dev/storage-accounting.md carries a per-format grid per arm, so the tempting shape is a row in each of the three. Prefer one shared passage that the three grids point at. The cause sits in internal/remote, not in any format, so three copies would each need editing when the cause moves, and a fourth remote format would need a fourth copy. AGENTS.md states the rule under "Comments Do Not Cite Process State": a pointer to a tracking work item appears once per obligation, because a paragraph copied to two places means the work item's landing has to find both, and whichever is missed survives as a stale reference.

The npm hosted grid's publish that fails after committing the tarball blob row is the closest existing shape.

What closes it

The options #910 lists for the npm hosted publish apply here unchanged, and whichever is taken should be taken once for all three arms rather than per format:

  1. Move the emit onto the CAS commit. This charges the namespace for bytes no artifact references yet.
  2. Let the retry tell "deduplicated against a referenced blob" from "deduplicated against an unreferenced one". This needs a reference probe the fill path does not have.
  3. Accept it and document it. This is what the code does today, undocumented.

Option 3 is the current behaviour, and reconciliation does correct it, so the divergence is bounded by reconciliation_interval wherever a pass runs.

Found while verifying !2262 (merged), which added the container arm's emit and so gave that arm a namespace delta to lose. The npm and Maven arms carried the window before it.

Related to #910, the same window on the npm hosted publish path.