npm packument bytes reach no counter, on write or on delete

Summary

An npm packument blob's bytes reach the storage counters on the write path, and they reach none of them on the two delete paths.

npm_metadata_files rows hold one packument blob per kind, at most three per package. recomputeNpmFilesSizeStmt unions those digests into the npm branch of the repository size walk, and the by-namespace shadow trigger fires on their blob_storage_blobs rows. Both recomputes therefore count packument bytes, and neither delete path carries them.

Three defects, one root. Two are the write half and one is the delete half. The two write-half defects are closed, and the section below records what closed them.

This issue was split out of #762, which bundled them with an ADR-007 amendment obligation that no merge request in this repository can discharge. It absorbs #761 (closed), which held the delete half on its own.

The write half: closed by emitRebuildCounters

Both gaps below are closed at 400ef5976 on main. emitRebuildCounters in internal/format/npm/packument_cache.go runs after each stamped kind's UpsertNpmMetadataFileForBlob commits, and it emits two deltas. NpmMetadataUpsertResult.RepoSizeDelta carries the repository-scoped movement, netted against the blob the repoint displaced, which closes gap 2. The namespace-scoped delta is the stored blob's size, and it is zero when the namespace already held the digest, which closes gap 3. The delete half is unaffected.

The two numbered gaps are kept below as the record of what the defect was. Copied unchanged from work item 762, where these were gaps 2 and 3.

2. The repository-scoped Δsize never carries packument bytes. ADR-007's Repository-level storage accounting reconciliation defines the npm repository footprint as npm_files in union with npm_metadata_files. recomputeNpmFilesSizeStmt implements both arms. No emit site covers the packument blob. UpsertNpmMetadataFileForBlob writes that row from the packument cache rebuild, and no emit site sits on that path.

3. The namespace-scoped Δdedup_size never carries packument bytes either. RecomputeDeduplicatedSizeBytes sums the blob_storage_blobs_by_namespace shadow table. A trigger populates that table for every blob row, packument blobs included.

Why the namespace gap matters most

The S22 spec names deduplicated_size_bytes the byte-based billing input. This counter under-reports, and it under-reports in the customer's favor. Nobody reports an error of that direction, so the gap can persist without a report from anyone.

The shape of the drift

UpsertNpmMetadataFileForBlob creates the new attachment and removes the previous one in one transaction. A package therefore holds one packument blob per kind at a time, and the blob is replaced rather than accumulated. Each rebuild is both a first attach of a new digest and a last detach of the old one, and the fast path sees neither.

The drift is not unbounded growth of one packument per publish. After a reconciliation pass writes the true value, the fast path moves away from it again on each later rebuild, by the size difference between the two packuments.

The delete half: two routes hard-delete the rows and carry none of their bytes

Copied unchanged from work item 761.

Two npm delete routes both tombstone and hard-delete inside one transaction, and only the tombstone half reaches repositories.size_bytes. This issue owns both arms:

• The management package DELETE, which hard-deletes at internal/datastore/npm_package_management_deleter.go:271.
• The protocol whole-package unpublish, which hard-deletes at internal/datastore/npm_package_unpublish_deleter.go:254.

Both routes make the same call, to NpmMetadataFileStore.DeleteNpmMetadataFiles, and one merge request can close both arms through it.

NpmPackageManagementDeleter.cascade marks the package, its versions, and its files, then calls removeCacheRows, which hard-deletes the package's npm_metadata_files rows and the blob_storage_attachments rows they held, inside the same transaction. The handler arm emits deleteCounterTargetTombstone, whose delta set is Δartifacts alone, because the footprint counters settle when the lifecycle purger reaps the marked rows.

NpmPackageUnpublishDeleter.cascade has the same shape. It marks the package with SoftDeleteNpmPackage and CascadeSoftDeleteNpmPackage, then hard-deletes the same rows with DeleteNpmMetadataFiles. Its handler emits Δsize = 0 at internal/format/npm/unpublish_package.go:263, on the same reasoning about the purger.

That reasoning holds for the marked rows and not for the hard-deleted ones. The mark is not what removes the packument's bytes from the size walk; the hard delete is. The purger can never settle them either, because the rows are already gone by the time it reaches the tombstone.

repositories.size_bytes therefore reads high by the packument blobs' size from the moment either delete commits until a reconciliation pass recomputes that scope.

Why this is not a delete-emit defect

S22 criterion 37's delta table has no row for a target that both tombstones and hard-deletes, so the gap sits in the criterion's model rather than in the emit that implements it. deleteCounterTarget cannot express the needed delta set, a tombstone that also frees bytes, so closing it is a widening rather than a one-line change.

What closing this needs

The write half needed an emit site and a datastore change, and both landed. UpsertNpmMetadataFileForBlob now returns *NpmMetadataUpsertResult, which carries the netted RepoSizeDelta and the blob the repoint displaced. The namespace half reads storage.StoreResult.Deduplicated as its first-attach predicate, which is the signal this paragraph proposed. What remains open here is the delete half.

The delete half needs a fourth deleteCounterTarget value, or a delta set carrying both amounts. NpmMetadataFileStore.DeleteNpmMetadataFiles already runs DELETE ... RETURNING and reports each removed row's attachment coordinates. It can report the bytes freed on the repository's last reference through the repoFreedBlobSizeTx helper, and the handler arm can carry that into the emit beside hiddenArtifacts.

Both delete arms reach that one helper, so one merge request can close both. The unpublish arm needs two more changes of its own: NpmPackageUnpublishDeleter.cascade takes no repository id today, and UnpublishPackage must carry the freed bytes back to its handler.

An amendment to S22 criterion 37's npm-package row belongs with the delete half, so the spec's table and the emit agree on what the arm owes.

Depends on !1997 (merged), and the dependency is the repository predicate

This section describes the write half, which has landed, and it is kept as the record. The repository predicate it asks for is in the tree, as the repoHeldBlob argument that NpmMetadataFileStore.netRepoSizeDelta takes.

The repository-scoped half of the write gap needs to know whether the repository already references a digest. !1997 (merged) introduces NpmFileStore.RepositoryHoldsBlob for exactly that question, on the npm publish path, and that method is on its branch rather than on main.

Merge order. Build the write half's repository arm on top of !1997 (merged) rather than beside it. Writing a second membership probe while that one is open produces two answers to one question, in one package.

#929 reports that RepositoryHoldsBlob fans out namespace-wide on the tarball digest. Whatever that issue settles about the query shape applies here too, because this issue's repository arm is its second consumer.

One caution for the implementer

ADR-007's Cleanup tasks puts attachment removal in the application's transaction, and ADR-007's repository accounting sentence says the counter decrements when garbage collection hard-deletes the last attachment.

For a packument the attachment goes inside UpsertNpmMetadataFileForBlob, which is the application's own transaction. A literal reading of that ADR-007 clause therefore never decrements, and the column climbs without bound. The S22 spec carries the workable rule, and it is what to follow here.

Work item 762 owns the amendment that reconciles the ADR text with the tree. This issue does not wait on it: the amendment moves the ADR's decrement clause for Container/OCI, and nothing here needs that clause to move first.

Scale and bound

At most three npm_metadata_files rows per package, each holding one packument blob.

RepositoryReconcileStore.WriteBackCounters takes an absolute sizeBytes and overrides the column, so no pass inherits a wrong value. The pass also selects namespaces by staleness rather than from the dirty set, so it reaches the repository whether or not an emit marked it.

reconciliation_interval is both the tick period and the staleness threshold, and config.example.yaml defaults it to 1h. A namespace reconciled at T is not a stale candidate before T + reconciliation_interval, and the next tick that re-selects it can be one interval later again. A wrong value therefore survives about one to two intervals.

Related

• #762 — the two ADR-007 handbook amendments this issue was split from.
• #910 — the other npm counter defect the split left standing. It needs a namespace attachment probe rather than the repository one, so it is a sibling rather than part of this.
• #929 — the query shape of the probe this issue's repository arm consumes.

Related to #515

---

Correction, 2026-09-04. The write half of this issue landed, and this description still described it as open. A review pass on !2262 (merged) found it. The edited passages are the ## Summary opening, the ## The write half heading and the paragraph under it, the write-half bullet under ## What closing this needs, and the marking paragraph under ## Depends on !1997. Every claim in those edits is measured at 400ef5976 on main. The delete half is unchanged, and this edit did not re-check it. The title still reads as though the write half is open.