feat(maven): the merged serve surface (S30 plan: 14/25)
Summary
Step 14 of the S30 plan: the merged package-level serve surface. internal/format/maven/virtual_serve.go serves the merged maven-metadata.xml: an input-derived validator (ETag over the canonical input encoding — version byte, then per contributing position the 4-byte big-endian position, 16-byte upstream id, 32-byte blob SHA-256) computed once every position's current row is known and before any blob is read, so a matching If-None-Match answers 304 after the database reads alone; exact Content-Length framing; HEAD running the full merge; If-Modified-Since not evaluated; a merged_partial response carrying no ETag; per-contributor last_downloaded_at bumps following each position's kind rules without incrementing downloads_count; an all-plugins-only collection served verbatim from its lowest-position contributor with no merge, and a collection that mixes shapes served as the merged <versioning> with the lowest-position contributor's root <plugins> beside it; and the serve entry taking Step 16a's cap slot for the admitted prefix, released on every terminal path, with positions past the admitted set recorded omitted_budget and the outcome merged_partial.
Step 13's Contribute is re-cut as the plan's Step 14 entry requires: a revalidation pass yielding each position's current-row digest, and a read pass running after the 304 decision (virtual_collect.go). The two merge metric families — maven_virtual_merge_duration_seconds and maven_virtual_merge_inputs — land in virtual_metrics.go beside Step 19a's winner-path families, registered in metrics.go.
Stacked on the Step 19a stack (!3014 (merged) + !3015 (merged), target branch mkhalifa/s30-maven-virtual-step-19a-event): this step extends virtual_metrics.go, which 19a creates. Rebase onto main once 19a merges.
Spec ambiguities resolved during implementation
Operator-resolved (all Option A) for the plugins verbatim arm, which the spec's merged-document sentences do not name:
- The plugins verbatim response carries the input-derived ETag and answers
If-None-Matchwith 304. - The plugins arm books neither merge metric family (no merge ran).
- Every contributing position bumps
last_downloaded_atunder its kind's rules.
Readings implementation-author chose (spec-consistent, flagged for review): a budget admitting zero positions answers 503 upstream_unavailable, and so does an all-absent or all-denied admitted prefix when the budget omitted a position (S30 now names this case); a read-pass failure of a stored blob omits the document, flips the token to failed, and forces merged_partial; if every contribution fails its read pass the read answers 503.
ADR conformance
- ADR-004 — no limit surface; the read-path rate limit is Step 16b's.
- ADR-005 — metadata serves honor the delivery axes; the merge reads stored blobs by content address.
- ADR-008 — cache serves open blobs by the row's content address, namespace-scoped.
- ADR-009 — no new route; the existing GET serves virtual repositories, writes stay gated.
- ADR-012 — merge-path usage events are Step 19b's; this step emits none.
Tests
go build ./..., go test ./... (69 packages), golangci-lint run (repo-pinned v2.13.0), and the comment-cap gate pass. The test-first commit (6bab74632, the documented --no-verify carve-out) carries the full spec-coverage table below; implementation and simplification commits ran the full hook chain.
Spec coverage
Spec: docs/specs/S30-maven-virtual.md
Acceptance criteria — Metadata merge
| # | Criterion | Tests |
|---|---|---|
| MM-1 | Union of versions | TestMavenMergeServe_GetServesTheMergedDocument (serve-level; the merge algorithm is the merge pass's own suite's) |
| MM-2 | Position-major ordering | Merge pass's suite. Not covered in this MR. |
| MM-3 | Newest timestamp wins the hints | Merge pass's suite. Not covered in this MR. |
| MM-4 | <lastUpdated> is the maximum |
Merge pass's suite. Not covered in this MR. |
| MM-5 | Coordinates come from the request | mvsRequireMergedBody, asserted in every served-document test |
| MM-6 | A remote contributor's freshness governs its input | TestMavenMergeServe_ConditionalRequest/a_required_revalidation_still_runs_ahead_of_the_304; the collection pass owns the per-outcome matrix |
| MM-7 | Best-effort under outage | TestMavenMergeServe_PartialMergeCarriesNoETag (omission arm; the cached-contribution arm is the collection pass's) |
| MM-8 | The merge memory budget bounds the contributor set | TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix, TestMavenMergeServe_AnAbsentPrefixWithOmittedPositionsIsUnavailable |
| MM-9 | An unhealthy remote contributes from cache only | Collection pass's suite. Not covered in this MR. |
| MM-10 | A hosted publish is visible through its stored document | Hosted-store integration. Not covered in this MR. |
| MM-11 | All-absent is a 404 | TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-absent_404 |
| MM-12 | All-failed is a 503 | TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-failed_503 |
| MM-13 | A data error vetoes the merge 404 | Verdict-to-status mapping is the dispatch step's (Step 17). Not covered in this MR. |
| MM-14 | A malformed input cannot fail the read | Parse-hardening; the merge pass's suite. Not covered in this MR. |
| MM-15 | A rejection on the committing read leaves the cache | S14 fill path. Not covered in this MR. |
| MM-16 | A plugins section is served from its lowest-position holder | TestMavenMergeServe_PluginsDocumentServedVerbatim, TestMavenMergeServe_AStreamedPluginsDocumentServedVerbatim, TestMavenMergeServe_AMixedCollectionServesBothSections, TestMergeMetadataContributions_PluginsSection |
| MM-17 | No sidecar fetch upstream | Step 18 (sidecar). Not covered in this MR. |
| MM-18 | Determinism | TestMavenMergeServe_DeterminismAcrossWinnerKinds |
| MM-19 | Merge statement count independent of upstream count | Datastore integration suite. Not covered in this MR. |
| MM-20 | Merge fan-out is bounded | Resolver-owned (S13). Not covered in this MR. |
| MM-21 | snapshot_metadata_always_revalidate does not reach the merge |
Collection pass's suite. Not covered in this MR. |
| MM-22 | Round trip | End-to-end integration. Not covered in this MR. |
| MM-23 | A winner change between dependent reads is visible | Step 18 (sidecar). Not covered in this MR. |
| MM-24 | FIPS short-circuit on the merged sidecar | Step 18 (sidecar). Not covered in this MR. |
| MM-25 | Merged metadata HEAD |
TestMavenMergeServe_HeadRunsTheFullMerge |
| MM-26 | A listed version may be unfetchable | Winner path. Not covered in this MR. |
| MM-27 | Per-contributor last_downloaded_at bumps, never downloads_count (design, "Sidecars, counters, and HEAD") |
TestMavenMergeServe_AccessBumpsFollowEachContributingKindRules, TestMavenMergeServe_A304BumpsOnlyTheHostedContributors, TestMavenMergeServe_AFailedBodyWriteHoldsBackTheRemoteBumps |
Acceptance criteria — Conditional requests
| # | Criterion | Tests |
|---|---|---|
| CR-1 | Merged-document validator; 304 after the database reads alone | TestMergeValidator_CanonicalEncoding, TestMavenMergeServe_ConditionalRequest |
| CR-2 | A partial merge carries no validator | TestMavenMergeServe_PartialMergeCarriesNoETag, TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix |
| CR-3 | Validator tracks inputs | TestMergeValidator_TracksInputs, TestMergeValidator_PositionOrderIsCanonical |
| CR-4 | Winner-path conditional request | Winner path (Step 17). Not covered in this MR. |
| CR-5 | A streamed cold fill carries no ETag | Winner path (Step 17). Not covered in this MR. |
| CR-6 | Sidecar conditional request | Step 18 (sidecar). Not covered in this MR. |
Error cases
| # | Condition | Tests |
|---|---|---|
| E-1 | Merge produced no document, an absence established: 404 not_found |
TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-absent_404 |
| E-2 | No absence established and no consulted position holds a cache entry: 503 upstream_unavailable |
TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath/an_all-failed_503 |
| E-3 | Merge read with some positions omitted: 200, merged_partial |
TestMavenMergeServe_PartialMergeCarriesNoETag, TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix |
| E-4 | Merge read, no document produced, data-error position in the list: 503 veto | Verdict-to-status mapping is the dispatch step's (Step 17). Not covered in this MR. |
| E-5 | All other rows (PUT 405, 429, winner-path and infrastructure rows) | Owned by the dispatch, winner-path, and rate-limit steps. Not covered in this MR. |
Security considerations
| # | Concern | Tests |
|---|---|---|
| S-1 | The merge parses upstream-controlled XML; the parser is the hardening surface | Parse hardening is the merge pass's suite. Not covered in this MR. |
| S-2 | The merge's contributor set is admitted against a memory budget | TestMavenMergeServe_BudgetOmitsThePositionsPastTheAdmittedPrefix, TestMavenMergeServe_CapSlotReleasedOnEveryTerminalPath |
| S-3 | Remaining considerations (authorization model, ordering, recursion guard, event volume, outbound surface, streaming bounds) | Owned by the resolution, dispatch, and rate-limit steps. Not covered in this MR. |
Merge metric families (this step's virtual_metrics.go extension)
| # | Concern | Tests |
|---|---|---|
| M-1 | maven_virtual_merge_duration_seconds / maven_virtual_merge_inputs carry no labels |
TestMavenVirtualMergeFamilies_CarryNoLabels |
| M-2 | Neither family takes prometheus.DefBuckets; named count/duration bucket families |
TestMavenVirtualMergeBuckets_AreNamedFamilies |
| M-3 | Registered through the composition root's prefixed registerer | TestRegisterMetrics_ExposedViaGatherer (extended) |
| M-4 | One merge books both families once; a 304 and the plugins arm book neither | TestMavenMergeServe_BooksTheMergeFamilies, TestMavenMergeServe_PluginsDocumentServedVerbatim |
Resolved spec ambiguities
- Conditional requests / plugins verbatim arm ETag -> Option A: the plugins
verbatim response carries the input-derived ETag and answers
If-None-Matchwith 304.TestMavenMergeServe_PluginsDocumentServedVerbatimpins the ETag; the 304 decision is the shared conditional path pinned byTestMavenMergeServe_ConditionalRequest. - Observability / plugins arm merge metrics -> Option A: the plugins arm
books neither
maven_virtual_merge_duration_secondsnormaven_virtual_merge_inputs(no merge ran). Pinned byTestMavenMergeServe_PluginsDocumentServedVerbatim. - Counters / plugins arm access bumps -> Option A: every contributing
position bumps
last_downloaded_atunder its kind's rules. Pinned byTestMavenMergeServe_PluginsDocumentServedVerbatim.
User documentation
No docs/user/ surface changes: virtual repositories are scoped as in development, and the end-of-plan docs pass owns the customer-facing write-up.
E2E scenario catalog
No docs/testing/ update: the end-to-end harness and e2e catalog are Step 20's, per the plan.
Related to #289 (closed)