S15: give a live npm remote metadata serve a validator
A proxied npm metadata read that is served live from the upstream carries no ETag, so the client it
answers holds no validator and cannot send If-None-Match on its next read. Giving that serve a
validator is a change to the serve shape, not a handler fix. This issue holds that design work.
Why a live serve carries no validator
npm.RemotePackumentHandler derives its validator from the content address of the bytes it is about
to serve: remote.ServeResult.BlobRef, rendered by remoteDocumentETag. That is the right validator
to advertise, because every dist.tarball is rewritten in-stream, so the upstream's own ETag names
a document the client never receives and must not be relayed.
On a remote.ServeFromUpstream there is no such address yet. The status line and headers are written
before io.Copy starts, and the CAS blob commits only once the upstream body has been read through
io.EOF. At header-write time the bytes being served have not been hashed, so no content address
exists. The handler passes the empty string and setRemoteDocumentHeaders omits the header.
What it costs
It compounds with the freshness directive. Every proxied response carries
Cache-Control: private, max-age=0, so a client is obliged to revalidate on each request. A client
answered by a live serve holds no validator, so its next request can send no If-None-Match, so the
conditional short-circuit that serveRemoteCachedConditional answers cannot fire, and it
re-downloads the whole document.
Two things widen that past one cold read per client per document:
- The gap re-opens on every refill. Once the freshness window lapses the read serves live again
through the
ServeFromUpstreamarm, which advertises no validator. - Wherever blobs are delivered by storage redirect, the cache serve is a
307carrying noETageither, so a client there never obtains a validator at all. Under proxy delivery a cache serve does carry one, which is what bounds the gap to the live serves.
The option
Make a validator exist before the response headers are written. Three shapes, none of them free:
- Hash the document while it streams, so its content address is known at header-write time.
- Buffer it, which stops
max_remote_packument_sizefrom being a bound on a stream. - Answer the live read with a redirect to the just-committed blob, which costs that path a second round trip.
!1929 (merged) does the equivalent for Maven remote metadata: drain the fill, then serve the committed copy.
It was filed as a type::feature MR with its own spec amendment and plan step, which is the shape
this would take for npm.
Scope
The packument and dist-tags proxies share the remote.Standalone serve shape, so both are in scope.
The tarball proxy is not: its Validators paragraph already states that a relay of a live upstream
body carries no validator, so it needs no spec change.
Relationship to the spec
While !1941 (merged) is open, the packument Serve paragraph in docs/specs/S15-npm-remote.md still says
both served variants carry "the ETag from the served body", which contradicts what ships. Once
!1941 (merged) merges, that paragraph states the rule per serve shape and the contradiction is gone, leaving
this issue holding the design work above and nothing else. ## Follow-ups in the same spec records
the option and its costs and points back here.
Raised in review of !1682 (merged).