Remote cache fill: a retried fill emits no namespace dedup bytes when an earlier attempt committed the blob
Summary
All three remote cache fills commit the payload blob before they write their cache
rows, and the storage-accounting emit is attached to the row write. A fill that
commits the blob and then fails leaves the blob_storage_blobs row behind with no
emit for it. The retry's Commit reports that digest as deduplicated, so the retry
emits Δdedup_size = 0 and namespace_statistics.deduplicated_size_bytes reads low
until a reconciliation pass recomputes it.
repositories.size_bytes is unaffected, because it keys on the repository's own
rows, which the failed attempt never wrote, so the retry charges it in full. The
asymmetry between the two columns is the shape to look for.
Where it applies
One cause, three consumers. internal/remote/fetch.go commits at
storage.Session.Commit and then calls CacheStore.UpsertCacheEntry, passing
stored.Deduplicated through as remote.CacheEntry.Deduplicated. Each arm reads
that field to decide its namespace delta:
internal/format/maven/remote_fill_emit.go:141internal/format/npm/npmremote/remote_fill_emit.go:97internal/format/oci/remote_fill_emit.go:45
None of the three remote grids in docs/dev/storage-accounting.md records the
window today.
Reproduction
Observed on the container arm against a kind=remote repository filling from
registry-1.docker.io, by failing the fill inside its own transaction (renaming the
column its blob probe reads) and then restoring it:
attempt 1 (fill fails): status 500
container_remote_manifests rows for the digest: 0
blob_storage_attachments rows for the digest: 0
blob_storage_blobs rows for the digest: 1 <- payload survives the rollback
attempt 2 (fill succeeds): status 200
ar:counter:repo_scoped:{<ns>:<repo>} artifacts 1 size 1025
ar:counter:namespace_scoped:{<ns>} components 1 dedup_size 0 <- never creditedA reconciliation pass then reported the gap and corrected it:
deduplicated_size_bytes 4224757 -> 4225782.
The injection is artificial; the window is not. Any failure inside the fill's own transaction reaches it, and so does a crash between the two commits.
Documenting it belongs in one place
docs/dev/storage-accounting.md carries a per-format grid per arm, so the tempting
shape is a row in each of the three. Prefer one shared passage that the three grids
point at. The cause sits in internal/remote, not in any format, so three copies
would each need editing when the cause moves, and a fourth remote format would need
a fourth copy. AGENTS.md states the rule under "Comments Do Not Cite Process
State": a pointer to a tracking work item appears once per obligation, because a
paragraph copied to two places means the work item's landing has to find both, and
whichever is missed survives as a stale reference.
The npm hosted grid's publish that fails after committing the tarball blob row is
the closest existing shape.
What closes it
The options #910 lists for the npm hosted publish apply here unchanged, and whichever is taken should be taken once for all three arms rather than per format:
- Move the emit onto the CAS commit. This charges the namespace for bytes no artifact references yet.
- Let the retry tell "deduplicated against a referenced blob" from "deduplicated against an unreferenced one". This needs a reference probe the fill path does not have.
- Accept it and document it. This is what the code does today, undocumented.
Option 3 is the current behaviour, and reconciliation does correct it, so the
divergence is bounded by reconciliation_interval wherever a pass runs.
Found while verifying !2262 (merged), which added the container arm's emit and so gave that arm a namespace delta to lose. The npm and Maven arms carried the window before it.
Related to #910, the same window on the npm hosted publish path.