S15: give a live npm remote metadata serve a validator

A proxied npm metadata read that is served live from the upstream carries no ETag, so the client it answers holds no validator and cannot send If-None-Match on its next read. Giving that serve a validator is a change to the serve shape, not a handler fix. This issue holds that design work.

Why a live serve carries no validator

npm.RemotePackumentHandler derives its validator from the content address of the bytes it is about to serve: remote.ServeResult.BlobRef, rendered by remoteDocumentETag. That is the right validator to advertise, because every dist.tarball is rewritten in-stream, so the upstream's own ETag names a document the client never receives and must not be relayed.

On a remote.ServeFromUpstream there is no such address yet. The status line and headers are written before io.Copy starts, and the CAS blob commits only once the upstream body has been read through io.EOF. At header-write time the bytes being served have not been hashed, so no content address exists. The handler passes the empty string and setRemoteDocumentHeaders omits the header.

What it costs

It compounds with the freshness directive. Every proxied response carries Cache-Control: private, max-age=0, so a client is obliged to revalidate on each request. A client answered by a live serve holds no validator, so its next request can send no If-None-Match, so the conditional short-circuit that serveRemoteCachedConditional answers cannot fire, and it re-downloads the whole document.

Two things widen that past one cold read per client per document:

  • The gap re-opens on every refill. Once the freshness window lapses the read serves live again through the ServeFromUpstream arm, which advertises no validator.
  • Wherever blobs are delivered by storage redirect, the cache serve is a 307 carrying no ETag either, so a client there never obtains a validator at all. Under proxy delivery a cache serve does carry one, which is what bounds the gap to the live serves.

The option

Make a validator exist before the response headers are written. Three shapes, none of them free:

  • Hash the document while it streams, so its content address is known at header-write time.
  • Buffer it, which stops max_remote_packument_size from being a bound on a stream.
  • Answer the live read with a redirect to the just-committed blob, which costs that path a second round trip.

!1929 (merged) does the equivalent for Maven remote metadata: drain the fill, then serve the committed copy. It was filed as a type::feature MR with its own spec amendment and plan step, which is the shape this would take for npm.

Scope

The packument and dist-tags proxies share the remote.Standalone serve shape, so both are in scope. The tarball proxy is not: its Validators paragraph already states that a relay of a live upstream body carries no validator, so it needs no spec change.

Relationship to the spec

While !1941 (merged) is open, the packument Serve paragraph in docs/specs/S15-npm-remote.md still says both served variants carry "the ETag from the served body", which contradicts what ships. Once !1941 (merged) merges, that paragraph states the rule per serve shape and the contradiction is gone, leaving this issue holding the design work above and nothing else. ## Follow-ups in the same spec records the option and its costs and points back here.

Raised in review of !1682 (merged).

Edited by Dzmitry (Dima) Meshcharakou