feat(npm): re-check the rebuild fence while the render is streaming
🎯 What this MR does
Re-checks the rebuild's invalidation fence while the render is streaming, so a superseded rebuild stops near where the writer landed instead of uploading both documents in full and only then being refused.
The fence protocol's guarantees do not change: the authoritative check is still the comparison the upsert makes under the package row's lock. What changes is how much work a refused rebuild does before it finds out.
Part 5 of 5 of Step 2 in the npm packument streaming generation plan. Depends on !1719 (merged), !1720 (merged), !1721 (merged) and !1722 (merged) — merge in that order.
| Part | MR | What it delivers |
|---|---|---|
| 2a | !1719 (merged) | the bounded reads the rebuild will walk |
| 2b | !1720 (merged) | blob writes routed through a hashing tee |
| 2c | !1721 (merged) | the rebuild streams into its blob sessions |
| 2d | !1722 (merged) | session lifetime and the two-session bound |
| 2e · this MR | fence re-checked per page |
🧩 Why streaming needs this
Streaming moved the cost of a refusal. On the materialized path the store step's pre-check ran before a single byte was written, so a rebuild superseded mid-render cost one SELECT and dropped an in-memory document. Streaming puts those bytes in a blob session as they are produced.
That is not a rare shape. enqueueRebuildAfterWrite deliberately retires the
singleflight key, so a burst of N writes to one package dispatches N rebuilds and
all but the last are superseded — and the writes that drive it,
PUT/DELETE .../dist-tags/{tag}, are the cheapest requests the API serves.
fencedVersionPager consults the fence once per page, and rebuildStreamedKinds
consults it once more before it opens any session at all. A session is not free
even when nothing is written to it: it inserts a row, does driver-side work,
allocates the staging buffer, and has to be canceled again. One indexed SELECT
per page is the price, against a page count the page size already bounds.
🔍 Two things worth a close read
A refusal is an error, never done=true. done terminates a well-formed
document at whatever row the walk had reached, and the rebuild would commit that
— a packument silently missing every version past the page it stopped on, served
for the whole cache TTL. Both new failure returns carry done=false, and the
suite pins it: flipping either to true fails these tests and nothing else in
the package.
Every detection site now logs through one helper. runRebuildWorker
suppresses its WARN for a supersede on the grounds that the detail was recorded
once already. With the fence consulted in three places that only holds if each of
them records it; otherwise the most common refusal is the one with no line naming
the package.
This does not close the window and must not be read as doing so. The fence can
still rotate between the last check and the commit, which is what
rebuildStoreKind's pre-check and the authoritative comparison in the upsert are
for. It converts the common case from "upload everything, then refuse" into
"refuse at the next page boundary".
📏 Size
433 reviewable LOC, within the 500-line guidance.
✅ Testing
- Unit,
-race, and integration suites green;golangci-lint0 issues, including a--build-tags=integrationrun. - npm conformance (
mise run conformance:npm) passes. - Both fence-read error branches are now covered, which nothing exercised
before: every
PackumentRebuildFenceHoldsdouble in the package returned a nil error, so the branch carrying thedone=falseproperty was unreachable in tests.
🧪 e2e scenarios
docs/testing/e2e/npm.md gains e2e.npm.discover.metadata-after-rebuild: after
the async rebuild has run, npm view returns the same versions, dist-tags and
tarball URLs, including for a package spanning more than one keyset page.
Related to #241