feat(npm): serve the merged virtual document, part 1/2 (S31 plan: 13/19)

David Fernandezrequested to merge
10io/s31-step13-part3-serve-surface-part1of2 into main

Split into 2 stacked MRs because the step is past the 500 reviewable LOC ceiling in docs/dev/development-model.md. Each part targets the previous one, and part 1 targets main. They are reviewed and merged bottom-up.

Part 1 of 2 of docs/plans/2026-08-21-s31-npm-virtual.md, Step 13: metadata merge and serve surface (Part 3: the serve surface).

Adds the function that serves the merged packument or dist-tags document of a virtual repository. No handler, route or composition-root wiring calls it yet; those are Steps 14 to 16.

  • Validator: a strong ETag over the merge inputs (a version byte, the target kind, then each contributing position, upstream repository and blob digest), so a matching If-None-Match is answered with a 304 after the database reads alone, with no blob opened.
  • Framing: headers commit on the first merged byte, the body is chunked with no Content-Length, and HEAD runs the same collect and merge into a discard so its status, ETag and outcome match GET.
  • Re-point: every tarball URL in the merged packument is rewritten to the virtual repository's base URL through an io.Pipe bridge from the merge into the tarball URL rewriter.
  • Cache-Control comes from the virtual repository's visibility, read from the repository finder.
  • Committed failures: a failure after the first byte aborts the connection (flush, then backdate the write deadline) so the client cannot read a truncated body as complete. remote_tarball.go extracts the flush-then-backdate core of abortRemoteTarballRelay into abortStreamedRelay; the tarball relay keeps its signature, log messages and behavior, and its existing tests cover it. A write-deadline kill is logged under its own message, and a recovered merge panic logs its type and stack.
  • Failures before the first byte go through writeVirtualReadError, which makes this its first production caller.

This part carries all the production code and the tests for the validator, the re-point, the per-kind headers and the committed-failure log causes. Part 2 carries the rest of the behavior tests. The two are reviewed together: the panic, abort and deadline paths in this part are also exercised end to end by tests in part 2.

Size

837 reviewable LOC, past the 500 ceiling: virtual_document_serve.go 340, remote_tarball.go 23, the virtual_document_serve.md sidecar 63, tests 411. The spec and plan edits are not counted. It does not split further because the production code is one serve entry point that its tests pin together, and the remaining behavior tests already sit in part 2.

Plan and spec amendments

  • Spec: the S31 (npm-virtual) validator encoding gains the target-kind byte after the version byte (0x01 full, 0x02 abbreviated, 0x03 dist-tags). Without it, the full and abbreviated packuments share one URL and can get the same validator over identical stored blobs (an upstream that ignores Accept), which breaks "equal validators mean byte-identical responses".
  • Plan, Step 13 Part 3: the file list now names remote_tarball.go and the virtual_document_serve.md sidecar. The fill-body Close test under ErrFillAbandoned is replaced by the test the collector allows: an abandoned fill twice and a canceled leader twice give the identical 503 with Retry-After, through the real collector. The serve reuses only redactedForLog (and writeIfClientClosed through writeVirtualReadError), and hosted entries are re-pointed with the tarball URL rewriter.
  • Follow-up before Steps 15 and 16: the merger in virtual_packument_merge.go closes its input readers inline instead of with defer, so a merge panic leaks the open reader. The serve now recovers such a panic into a 500. The follow-up is tracked on work items 896 and 897.
  • No e2e catalog scenario is added or affected: nothing is reachable over HTTP until the handler steps.
  • No user documentation is affected: no customer-visible surface changes.

Spec: S31 (npm-virtual) Plan: docs/plans/2026-08-21-s31-npm-virtual.md

Related to #894 (closed)

Stacked MRs (review/merge bottom-up)

Edited by David Fernandez

Merge request reports