test(remote): Fetch edge cases, integration floor, CI wiring (S13 Step 7, part 4/4)

David Fernandezrequested to merge
10io/s13-step7-part4of4 into main

📦 What this MR does

Part 4 of 4 of S13 Step 7 (Fetch cache-fill pipeline): the error-injection/edge-case unit floor, the integration floor, and its CI wiring.

  • Nineteen unit tests: transport/builder/session-open failures, BodyKind + MaxBodyBytes pass-through to the doer (fail-small default pinned), Transform applied to both caller and cache, post-EOF upsert and commit failures surfacing from Read wrapped in ErrCacheFill (commit failure never triggers a redundant Cancel), the post-EOF staging-flush failure carrying that same sentinel, mid-stream staging write/short-write discard (including the over-buffer path that drives more than one session write), self-terminated-session discard (no misleading joined error), Cancel-error propagation from Read and from Close, post-Close idempotency (ErrFillAbandoned, no phantom io.EOF), the unstorable-ETag drop on every rejection dimension (over-long, injection bytes, invalid UTF-8), the vanished-row 304 degrading to one unconditional re-fetch with If-None-Match stripped (ErrCacheEntryNotFound never escapes Fetch), and the surviving-content-coding rejection on a 200 (br/deflate/zstd/multi-coding list rejected as FetchError, identity and empty codings staying on the stream path).
  • Two integration tests against a real Postgres-backed BlobStore/Session (upstream faked at the UpstreamDoer seam): a full miss/200 fill committing a real blob row and upserting the cache entry, and a mid-stream failure leaving no committed blob (asserted for both the full and truncated-prefix digests) and no cache entry.
  • .gitlab-ci.yml: wires internal/remote into the test:integration job (non-recursive on purpose — upstreamhttp and remotetest carry no integration-tagged files; a rationale comment matches the neighboring entries).

📊 Spec coverage (whole stack)

The table below maps the spec's acceptance criteria, error cases, and security considerations for Step 7's scope to tests; the named TestFetcher_* tests land in part 3 (acceptance paths) and this part (error/edge cases and integration). Rows owned by other S13 steps are marked with their owning step.

Criterion Tests / Owner
Miss, upstream 200 TestFetcher_Fetch_Miss200_TeesIntoCacheAndStreams (part 3), TestFetcher_Fetch_Integration_Miss200_CommitsRealBlobAndUpsertsCacheEntry (this part); composed into the standalone flow by Step 11
Stale hit, etag matches TestFetcher_Fetch_ConditionalRequest_304ReturnsFreshFromCache (part 3); Step 11 composes
Stale hit, etag differs TestFetcher_Fetch_ConditionalRequest_ETagDiffers_FullRefetch (part 3); Step 11 composes
Winner fetch fails mid-stream TestFetcher_Fetch_MidStreamReadError_CancelsStaging, TestFetcher_Fetch_EarlyClose_CancelsStaging (part 3), TestFetcher_Fetch_Integration_MidStreamFailure_DiscardsRealStaging (this part); virtual no-fall-through is Step 12
Miss, upstream 401/404/5xx TestFetcher_Fetch_NonSuccessStatus_ReturnsFetchError (part 3); verbatim propagation is Step 11
Stale hit, no stored ETag / cache_validity_hours=0 / digest-keyed hit Step 6 (RevalidationFor, merged)
Transport failure paths TestFetcher_Fetch_TransportFailure_ReturnsError (this part); cache fallback composition is Steps 11/12
Body cap / decompression / redirect / header-injection covers Steps 3-5 (merged); propagate through Fetch via the UpstreamDoer error path
Single-flight coalescing Step 8
Virtual resolution scenarios, allow/deny, health Steps 10/12/14/15
Error-payload hygiene (SafeHeaders allowlist) Step 2 (merged); SafeHeaders asserted on both FetchError and FetchStreamHit paths (parts 3/this part)
Upstream content trust (no format validation on proxy path) Byte-for-byte fidelity asserted in the miss/200 and Transform tests (part 3/this part)
Cache-infrastructure failure classification ErrCacheFill/ErrFillAbandoned sentinel tests (this part); consumed by Steps 11/12 SLI classification

⚙️ Why stacked

Step 7's full diff is ~2.7k reviewable LoC, so it ships as 4 stacked MRs, reviewed and merged bottom-up. This is the stack top: with it merged, the branch content is complete.

🔗 References

  • Plan: docs/plans/2026-07-16-s13-virtual-remote-foundation.md — Step 7
  • Spec: docs/specs/S13-virtual-remote-foundation.md — Fetch, cache freshness model

🔬 e2e scenarios

No scenario added or affected: S13 remote fetch has no e2e catalog yet (test and CI-wiring MR).

Related to #328 (closed)

📚 Stacked MRs (review/merge bottom-up)

Edited by David Fernandez

Merge request reports