feat(maven): the merged serve surface (S30 plan: 14/25)

Summary

Step 14 of the S30 plan: the merged package-level serve surface. internal/format/maven/virtual_serve.go serves the merged maven-metadata.xml: an input-derived validator (ETag over the canonical input encoding — version byte, then per contributing position the 4-byte big-endian position, 16-byte upstream id, 32-byte blob SHA-256) computed once every position's current row is known and before any blob is read, so a matching If-None-Match answers 304 after the database reads alone; exact Content-Length framing; HEAD running the full merge; If-Modified-Since not evaluated; a merged_partial response carrying no ETag; per-contributor last_downloaded_at bumps following each position's kind rules without incrementing downloads_count; an all-plugins-only collection served verbatim from its lowest-position contributor with no merge, and a collection that mixes shapes served as the merged <versioning> with the lowest-position contributor's root <plugins> beside it; and the serve entry taking Step 16a's cap slot for the admitted prefix, released on every terminal path, with positions past the admitted set recorded omitted_budget and the outcome merged_partial.

Step 13's Contribute is re-cut as the plan's Step 14 entry requires: a revalidation pass yielding each position's current-row digest, and a read pass running after the 304 decision (virtual_collect.go). The two merge metric families — maven_virtual_merge_duration_seconds and maven_virtual_merge_inputs — land in virtual_metrics.go beside Step 19a's winner-path families, registered in metrics.go.

Stacked on the Step 19a stack (!3014 (merged) + !3015 (merged), target branch mkhalifa/s30-maven-virtual-step-19a-event): this step extends virtual_metrics.go, which 19a creates. Rebase onto main once 19a merges.

Spec ambiguities resolved during implementation

Operator-resolved (all Option A) for the plugins verbatim arm, which the spec's merged-document sentences do not name:

  1. The plugins verbatim response carries the input-derived ETag and answers If-None-Match with 304.
  2. The plugins arm books neither merge metric family (no merge ran).
  3. Every contributing position bumps last_downloaded_at under its kind's rules.

Readings implementation-author chose (spec-consistent, flagged for review): a budget admitting zero positions answers 503 upstream_unavailable, and so does an all-absent or all-denied admitted prefix when the budget omitted a position (S30 now names this case); a read-pass failure of a stored blob omits the document, flips the token to failed, and forces merged_partial; if every contribution fails its read pass the read answers 503.

ADR conformance

  • ADR-004 — no limit surface; the read-path rate limit is Step 16b's.
  • ADR-005 — metadata serves honor the delivery axes; the merge reads stored blobs by content address.
  • ADR-008 — cache serves open blobs by the row's content address, namespace-scoped.
  • ADR-009 — no new route; the existing GET serves virtual repositories, writes stay gated.
  • ADR-012 — merge-path usage events are Step 19b's; this step emits none.

Tests

go build ./..., go test ./... (69 packages), golangci-lint run (repo-pinned v2.13.0), and the comment-cap gate pass. The test-first commit (6bab74632, the documented --no-verify carve-out) carries the full spec-coverage table below; implementation and simplification commits ran the full hook chain.

Spec coverage

Spec: docs/specs/S30-maven-virtual.md

Acceptance criteria — Metadata merge

# Criterion Tests
MM-1 Union of versions TestMavenMergeServe_GetServesTheMergedDocument (serve-level; the merge algorithm is the merge pass's own suite's)
MM-2 Position-major ordering Merge pass's suite. Not covered in this MR.
MM-3 Newest timestamp wins the hints Merge pass's suite. Not covered in this MR.
MM-4 <lastUpdated> is the maximum Merge pass's suite. Not covered in this MR.
MM-5 Coordinates come from the request mvsRequireMergedBody, asserted in every served-document test
MM-6 A remote contributor's freshness governs its input TestMavenMergeServe_ConditionalRequest/a_required_revalidation_still_runs_ahead_of_the_304; the collection pass owns the per-outcome matrix
MM-7 Best-effort under outage TestMavenMergeServe_PartialMergeCarriesNoETag (omission arm; the cached-contribution arm is the collection pass's)
MM-8 The merge memory budget bounds the contributor set TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix, TestMavenMergeServe_AnAbsentPrefixWithOmittedPositionsIsUnavailable
MM-9 An unhealthy remote contributes from cache only Collection pass's suite. Not covered in this MR.
MM-10 A hosted publish is visible through its stored document Hosted-store integration. Not covered in this MR.
MM-11 All-absent is a 404 TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-absent_404
MM-12 All-failed is a 503 TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-failed_503
MM-13 A data error vetoes the merge 404 Verdict-to-status mapping is the dispatch step's (Step 17). Not covered in this MR.
MM-14 A malformed input cannot fail the read Parse-hardening; the merge pass's suite. Not covered in this MR.
MM-15 A rejection on the committing read leaves the cache S14 fill path. Not covered in this MR.
MM-16 A plugins section is served from its lowest-position holder TestMavenMergeServe_PluginsDocumentServedVerbatim, TestMavenMergeServe_AStreamedPluginsDocumentServedVerbatim, TestMavenMergeServe_AMixedCollectionServesBothSections, TestMergeMetadataContributions_PluginsSection
MM-17 No sidecar fetch upstream Step 18 (sidecar). Not covered in this MR.
MM-18 Determinism TestMavenMergeServe_DeterminismAcrossWinnerKinds
MM-19 Merge statement count independent of upstream count Datastore integration suite. Not covered in this MR.
MM-20 Merge fan-out is bounded Resolver-owned (S13). Not covered in this MR.
MM-21 snapshot_metadata_always_revalidate does not reach the merge Collection pass's suite. Not covered in this MR.
MM-22 Round trip End-to-end integration. Not covered in this MR.
MM-23 A winner change between dependent reads is visible Step 18 (sidecar). Not covered in this MR.
MM-24 FIPS short-circuit on the merged sidecar Step 18 (sidecar). Not covered in this MR.
MM-25 Merged metadata HEAD TestMavenMergeServe_HeadRunsTheFullMerge
MM-26 A listed version may be unfetchable Winner path. Not covered in this MR.
MM-27 Per-contributor last_downloaded_at bumps, never downloads_count (design, "Sidecars, counters, and HEAD") TestMavenMergeServe_AccessBumpsFollowEachContributingKindRules, TestMavenMergeServe_A304BumpsOnlyTheHostedContributors, TestMavenMergeServe_AFailedBodyWriteHoldsBackTheRemoteBumps

Acceptance criteria — Conditional requests

# Criterion Tests
CR-1 Merged-document validator; 304 after the database reads alone TestMergeValidator_CanonicalEncoding, TestMavenMergeServe_ConditionalRequest
CR-2 A partial merge carries no validator TestMavenMergeServe_PartialMergeCarriesNoETag, TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix
CR-3 Validator tracks inputs TestMergeValidator_TracksInputs, TestMergeValidator_PositionOrderIsCanonical
CR-4 Winner-path conditional request Winner path (Step 17). Not covered in this MR.
CR-5 A streamed cold fill carries no ETag Winner path (Step 17). Not covered in this MR.
CR-6 Sidecar conditional request Step 18 (sidecar). Not covered in this MR.

Error cases

# Condition Tests
E-1 Merge produced no document, an absence established: 404 not_found TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-absent_404
E-2 No absence established and no consulted position holds a cache entry: 503 upstream_unavailable TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-failed_503
E-3 Merge read with some positions omitted: 200, merged_partial TestMavenMergeServe_PartialMergeCarriesNoETag, TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix
E-4 Merge read, no document produced, data-error position in the list: 503 veto Verdict-to-status mapping is the dispatch step's (Step 17). Not covered in this MR.
E-5 All other rows (PUT 405, 429, winner-path and infrastructure rows) Owned by the dispatch, winner-path, and rate-limit steps. Not covered in this MR.

Security considerations

# Concern Tests
S-1 The merge parses upstream-controlled XML; the parser is the hardening surface Parse hardening is the merge pass's suite. Not covered in this MR.
S-2 The merge's contributor set is admitted against a memory budget TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix, TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath
S-3 Remaining considerations (authorization model, ordering, recursion guard, event volume, outbound surface, streaming bounds) Owned by the resolution, dispatch, and rate-limit steps. Not covered in this MR.

Merge metric families (this step's virtual_metrics.go extension)

# Concern Tests
M-1 maven_virtual_merge_duration_seconds / maven_virtual_merge_inputs carry no labels TestMavenVirtualMergeFamilies_CarryNoLabels
M-2 Neither family takes prometheus.DefBuckets; named count/duration bucket families TestMavenVirtualMergeBuckets_AreNamedFamilies
M-3 Registered through the composition root's prefixed registerer TestRegisterMetrics_ExposedViaGatherer (extended)
M-4 One merge books both families once; a 304 and the plugins arm book neither TestMavenMergeServe_BooksTheMergeFamilies, TestMavenMergeServe_PluginsDocumentServedVerbatim

Resolved spec ambiguities

  • Conditional requests / plugins verbatim arm ETag -> Option A: the plugins verbatim response carries the input-derived ETag and answers If-None-Match with 304. TestMavenMergeServe_PluginsDocumentServedVerbatim pins the ETag; the 304 decision is the shared conditional path pinned by TestMavenMergeServe_ConditionalRequest.
  • Observability / plugins arm merge metrics -> Option A: the plugins arm books neither maven_virtual_merge_duration_seconds nor maven_virtual_merge_inputs (no merge ran). Pinned by TestMavenMergeServe_PluginsDocumentServedVerbatim.
  • Counters / plugins arm access bumps -> Option A: every contributing position bumps last_downloaded_at under its kind's rules. Pinned by TestMavenMergeServe_PluginsDocumentServedVerbatim.

User documentation

No docs/user/ surface changes: virtual repositories are scoped as in development, and the end-of-plan docs pass owns the customer-facing write-up.

E2E scenario catalog

No docs/testing/ update: the end-to-end harness and e2e catalog are Step 20's, per the plan.

Related to #289 (closed)

Edited by Moaz Khalifa

Merge request reports

Loading
Loading