feat(oci): serve virtual resolution winners (S32 plan: 12/19)

What This Does

Serve virtual manifest and blob resolution winners.

The serve branches on winner kind to choose which row supplies the response. It applies manifest content negotiation and keeps the blob delivery-mode behavior. A delivered read records its download on the position that supplied it.

Decisions A Reviewer Should Check First

  • A hosted winner serves from its Phase 1 blob reference and the media type its position carries. A remote winner serves a warm blob from its Phase 1 blob reference too, and a cold read or a manifest from its cache row, through the RemoteReadCache the calling route supplies. The serve refuses a remote winner with no cache surface before it writes any header.
  • A warm remote blob hit whose row was deleted between Phase 1 and the serve answers 200 instead of 500, because the serve no longer reads that row.
  • A virtual manifest always streams. A cold manifest drains its fill first, then negotiates against the media type the commit stored.
  • A stored blob uses the configured delivery mode. A cold full blob GET streams chunked while the fill is active.
  • Cold HEAD and ranged blob reads drain the fill, then serve the committed row.
  • A failed drain goes through the virtual read error mapping. A digest mismatch answers 503 with its fixed Retry-After. An upstream transport failure (remote.FallbackEligible) answers 503 with the health-sweep Retry-After. A local fault answers 500.
  • A failed streamed blob fill backdates the response deadline, so net/http cannot frame a partial response as complete. A client hangup does not.
  • A client that leaves during a failed call or a failed drain gets 499, not 500 or 503. The serve's own faults, and a named server fault behind a hangup, keep their Error record.
  • A delivered read records its download on the supplying position, per !2907 (merged). A hosted supplier books downloads_count through CounterSink.emitDownload on its own repositories row. A remote supplier bumps last_downloaded_at through S16's download signals. The virtual repository books nothing. A 404, a 416, a failed read, or a cut-short transfer records nothing.

#1430 (closed): the drain-failure fallback is not in this MR

#1430 (closed) is decided: a drain failure before any header takes the cache fallback on the winner's position, and answers 503 when nothing is cached. This MR ships the 503 half, which is correct under that decision. The fallback needs a resolver entry point from a foundation MR, so a follow-up MR adds it to drainFill once that entry point merges.

Spec Coverage

Requirement Tests
Equal headers for hosted and remote manifest and blob winners TestVirtualServe_HostedAndRemoteWinnersAnswerEqualHeaders
Warm and cold manifest negotiation, including mismatch, on the committed media type TestVirtualServe_Manifest
Manifest delivery never redirects, and a manifest HEAD matches its GET TestVirtualServe_Manifest
Warm blob delivery, range, and HEAD behavior per winner and delivery mode TestVirtualServe_BlobDelivery
Cold full GET streams, and cold HEAD or range drains then serves TestVirtualServe_BlobDelivery
HEAD ignores Range, 416, and a range re-opened past byte zero TestVirtualServe_BlobRanges
A failed cold drain answers 503 with its class's Retry-After: a digest mismatch and a connection reset TestVirtualServe_FailedDrainIsMappedThroughThePolicy
A failed streamed blob fill interrupts the response TestVirtualServe_StreamedBlobFailureInterruptsResponse, TestVirtualServe_RefusedStreamIsNotFramedAsCompleteToTheClient
A delivered read records one download on the supplying position; a failed read records none every table above, through requireDownload and requireNoDownload
Hosted and remote virtual failure fields TestVirtualServe_FailureLogsWinnerIdentifiers
The serve and the failure log read the winner, not the first position TestVirtualServe_ServesTheWinnerPastTheFirstPosition, TestVirtualServe_FailureLogsWinnerIdentifiers
A client that leaves during a failed call answers 499, and a canceled upstream read does not name the upstream TestVirtualServe_ClientLeavingDuringAFailedCallIsNotInternal, TestVirtualServe_UpstreamReadAfterTheClientLeftIsNotAnUpstreamFault
An unserved action answers 501 and closes the stream TestVirtualServe_UnservedActionIsNotImplemented
Each answer's header set is closed, and a cold fill relays no upstream header TestVirtualServe_BlobDelivery, TestVirtualServe_Manifest
Accept admits across several field lines, the wildcard, and a q-valued list TestVirtualServe_Manifest
A drain whose client left answers 499, keeping the Error record of a named fault TestVirtualServe_FailedDrainIsMappedThroughThePolicy
The assembly sets RemoteRepositoryID for a bound remote and leaves it nil for a hosted row TestVirtualUpstreamAssembly_ClassifiesEachKindAndFormat

Files outside Step 12's plan entry

  • internal/format/oci/virtual_assembly.go and virtual_assembly_test.go: VirtualPosition carries RemoteRepositoryID, which the failure log reads.
  • docs/dev/storage-accounting.md: the virtual read is a new download-booking site, so the emit table gets its row.
  • internal/format/oci/remote_download.md: the remote_download.go header's content moved to this sidecar to fit the comment cap once the header changed.

!2904 (merged) adds the first two and the download record to Step 12's entry. It does not name the sidecar.

Size

The diff adds 698 source lines, 1,423 test lines, and 42 doc lines against the merge base. The plan estimates ~470 source and ~1,120 test. The manifest and blob halves share the winner-kind branch and the response construction, so splitting them lands that branch twice. The plan's seam interrogation reaches the same answer.

Checks

  • env GOOGLE_APPLICATION_CREDENTIALS="$HOME/.cache/ar-dummy-gcp/adc.json" go test -race ./internal/format/oci/ ./internal/virtual/
  • golangci-lint run --max-same-issues=0 --max-issues-per-linter=0 --uniq-by-line=false ./internal/format/oci/, with and without --build-tags=integration
  • Integration tests were not run. This MR adds no integration test.
  • No e2e scenario is added or affected. Step 18 owns the virtual repository catalog sections.
  • No user documentation is affected. No route reaches this serve until Steps 15 and 16.

Stack and merge order

  • This draft MR targets main. Its Step 11 base, !2842 (merged), has merged.
  • Merge after !2907 (merged), the S32 spec amendment that defines the download record, and after !2904 (merged), the batched docs(plans) MR that adds it to Step 12.
  • The Step 15 and Step 16 builders must hand virtualServeHandler.counters the process's CounterSink. Until they do, a hosted supplier books no download. !2904 (merged) books that wiring on both steps.

Related to #291 (closed)

Edited by Radamanthus Batnag

Merge request reports

Loading
Loading