feat(maven): remote upstream lookup and prober (S30 plan: 6/25)
Summary
Implements Step 6 of the S30 plan: the batched cache reads over the remote
cache tables, the Phase 1 adapter, and the bare-HEAD prober a remote position
probes through. Depends on: — per the plan, so it targets main directly.
MavenVirtualRemoteLookupStore runs one statement per document class —
package-level metadata and version-level files — over maven_remote_files
joined to maven_remote_packages (and maven_remote_versions for the
version-level class), keyed on namespace_id equality on every joined table
plus the positions as one IN list on maven_remote_packages
(maven_remote_repository_id is not a column of maven_remote_files), split
on maven_remote_version_id, with soft_deleted_at IS NULL on files,
versions, and packages, and each row's freshness verdict computed per S14's
model. MavenVirtualRemoteUpstream binds one position's prefetched verdict.
NewMavenVirtualRemoteProber composes the bare-HEAD health probe: a lazy per-call
builder in the flightRequestBuilder shape reads the credential inside the
call, so a denied or unhealthy position never materializes its upstream
credential. A failed credential read is positional: it surfaces the new
remote.ErrCredentialsUnreadable marker, which probeErrorIsPositional
admits beside the two durable markers — the read never consulted the
upstream, so the position fails with the resolution's data-error flag set,
and a sibling's ProbeMiss cannot answer 404 for an artifact this
position may still hold. The branch is rebased on today's main, which
books these foundation files in the step's Files list.
Reviewable LOC: 3,662 insertions — past the 500 ceiling (the guardrail's own count: it excludes only vendored, generated, and binary files, so tests and docs count toward it). By file group: the store and its query names 300, the adapter and prober 244, the foundation marker and its resolver admission 31, tests 2,800, sidecar docs 287 (3,662 insertions, 14 deletions, measured at the branch tip). A split would not help: the store, adapter, and prober are one seam's three parts — the prober's builder consumes the store's projected URL and the adapter consumes both reads — and the foundation marker must land with its only consumer, so a source-first MR would be a stacked dependency on a ~300-line first part.
References: S30 spec · S30 plan (see "Step 6: remote upstream lookup and prober")
Governing ADRs
- ADR-001 (organizations as anchor) —
namespace_idequality leads on every joined table in both reads, and the credential read keys on(namespace_id, repository_id); the integration suite asserts cross-namespace rows are unreachable. - ADR-004 (data and application limits) — the reads enforce no limit and bypass none; the 20-upstream cap is cited as the position list's bound, and the per-probe credential-read cost is recorded in the adapter sidecar.
- ADR-005 (artifact delivery mode) — the prober issues a
HEADhealth probe only; the redirect-or-proxy decision belongs to the serve path of later steps. - ADR-007 (database schema) — namespace-leading predicates and
soft_deleted_at IS NULLon all three cache tables match the ADR's remote-cache query examples; themaven_remote_version_id IS NULL/IS NOT NULLsplit matches the two partial unique indexes, and the join-order guard pins single-partition pruning per table. No credential column (encrypted_*,tmp_plaintext_*) is projected by either statement. - ADR-008 (content-addressable storage) — verdicts key on the stored
blob_sha256content address; the adapter validates the 32-byte digest shape before it can become a serve-path digest. - ADR-009 (API design) — no route, endpoint, or URL shape added or altered.
- ADR-012 (usage data) — no usage events emitted; the virtual pull stream is Step 17's.
- Mirror freshness:
scripts/adr-freshness.shreports three upstream amendments not yet synced — the two ADR-007 amendments ("two container index lookups gain their order column"; "maven_filesgains the metadata tombstone discovery index") touch tables and listings this diff neither reads nor indexes, and ADR-022 (namespace UUID as the authorization resource identifier) is not cited by this diff.
Testing
Mirrors npm's step-6 suites per the mirroring rule, with no dropped subtest
that has a Maven analog (npm's per-kind Target URI dispatch has no Maven
surface; the bare path is covered by the bodyless-HEAD test):
- Statement-rendering unit tests assert the version-anchor split, namespace
equality on every joined table, the coordinate binds as bound literals
(
group_id,artifact_id), freshness-class column selection, and the window staying out of the row predicates. - The guards suite rejects every bind that would silently read as absence — zero UUIDs, empty coordinates, nil context, nil client — with its own sentinel.
- The integration suite covers both document classes: fresh, stale, and
missing verdicts per position in one call; per-shape soft-delete negatives
on files, versions, and packages; the revalidation opt-in and zero-pin
arms; one statement per class at 2 and at 20 positions; cross-namespace
and cross-group exclusion; and the join-order guard with its
forced-failure pass (measured plans recorded in
maven_virtual_remote_lookup_join_order.md). - The prober suite pins the bodyless
HEADat the projected base, the stored Basic pair or noAuthorizationat all, the fail-small metadata cap, one credential read per probe and never at construction, the credential read sharing the probe deadline, and the positional marker for all four of its cases: two read-side causes and the two durable pass-throughs. - The resolver fixtures pin the data-error booking: a credential-failed
position alongside a healthy one still resolves; alongside only a
ProbeMissit answers unavailable rather than not-found; under a departed caller the cancellation books the abandonment.
Integration suite run locally against a CI-tuned PostgreSQL; golangci-lint
run with --build-tags=integration and the documented flag set, which CI
lint cannot see. go test ./internal/datastore/ ./internal/format/maven/
green after the rebase onto today's main.
A five-category pre-push coherence check (plan adherence, acceptance
criteria, spec adherence, ADR adherence, dev guidelines) ran on this
branch: no blockers. The advisories it raised are fixed on this branch
(the group-id coverage gap, the two prose rewords, the maintidx
confirmation), except the lll banner claim — measured as a byte-vs-rune
counting error, no finding raised — and npm twin's matching sidecar
sentence, left for a standalone chore MR.
e2e scenarios
No scenario added or affected: no route serves virtual reads until the dispatch steps land, so this MR alone changes no client-visible behavior.
User documentation
No user-facing surface changes: every surface here — the batched cache
reads, the Phase 1 adapter, and the prober — is internal composition no
route serves until the dispatch steps land, so docs/user/** is
unaffected by this MR.
Related to #289
Database Review Evidence
Queries
Note
Plans are from EXPLAIN (ANALYZE, BUFFERS) against an ephemeral
PostgreSQL 17 container (matching GL_PG_CURR_VERSION from
.gitlab-ci-other-versions.yml), with synthesized seed data rolled
back per query and the container torn down at the end of the run.
Numbers reflect moderate cardinality and do not capture
production-scale effects. See
Database review evidence
for seed sizing, methodology, and the anomalies the skill flags.
Expand each row's details for the seed shape, rendered SQL, bound args,
and raw plan.
| Method | Plan node | Index | Rows (plan / actual) | Cost | Time | Buffers (hit / read) | Partitions |
|---|---|---|---|---|---|---|---|
datastore.ByPositions._VersionEmpty |
Nested Loop | maven_remote_packages_p41_namespace_id_maven_remote_reposit_idx, maven_remote_files_p41_namespace_id_maven_remote_package_i_idx1 |
1 / 2 | 21.98 | 0.026ms | 13 / 0 | 1/64, 1/64, 1/64 |
datastore.ByPositions._VersionNonEmpty_Metadata |
Nested Loop | maven_remote_packages_p19_namespace_id_maven_remote_reposit_idx, maven_remote_versions_p19_namespace_id_maven_remote_packag_idx4, maven_remote_files_p19_namespace_id_maven_remote_version_id_idx |
1 / 2 | 30.30 | 0.034ms | 19 / 0 | 1/64, 1/64, 1/64 |
datastore.ByPositions._VersionNonEmpty_Artifact |
Nested Loop | maven_remote_packages_p59_namespace_id_maven_remote_reposit_idx, maven_remote_versions_p59_namespace_id_maven_remote_packag_idx4, maven_remote_files_p59_namespace_id_maven_remote_version_id_idx |
1 / 2 | 30.30 | 0.033ms | 19 / 0 | 1/64, 1/64, 1/64 |
datastore.ByPositions._VersionEmpty
Summary: Plan matches the method's intent: the packages composite index drives a nested loop — the position IN list renders as = ANY — and each surviving package probes the files unique index on (namespace_id, maven_remote_package_id), with the freshness verdict projected, not filtered. The bound namespace_id prunes every scanned table to one of 64 partitions. The repositories arm is a Seq Scan over that partition's 2-row slice — optimal at this cardinality, since remote repositories per namespace are few. No anomalies.
Seed shape: namespaces=1, repositories=2, maven_remote_repositories=2, blob_storage_blobs=1, blob_storage_attachments=1, maven_remote_packages=10002, maven_remote_files=10002 (one metadata file per package, maven_remote_version_id NULL)
Rendered SQL:
SELECT maven_remote_packages.maven_remote_repository_id AS "maven_remote_packages.maven_remote_repository_id",
maven_remote_files.blob_sha256 AS "maven_remote_files.blob_sha256",
maven_remote_files.upstream_etag AS "maven_remote_files.upstream_etag",
(maven_remote_files.upstream_checked_at > (NOW() - (INTERVAL '1 HOUR' * maven_remote_repositories.metadata_cache_validity_hours))) AS "fresh"
FROM public.maven_remote_files
INNER JOIN public.maven_remote_packages ON ((maven_remote_packages.id = maven_remote_files.maven_remote_package_id) AND (maven_remote_packages.namespace_id = maven_remote_files.namespace_id))
INNER JOIN public.maven_remote_repositories ON ((maven_remote_repositories.id = maven_remote_packages.maven_remote_repository_id) AND (maven_remote_repositories.namespace_id = maven_remote_packages.namespace_id))
WHERE ((((maven_remote_files.namespace_id = $1::uuid) AND (maven_remote_files.maven_remote_version_id IS NULL)) AND (maven_remote_files.file_name = $2::text)) AND (maven_remote_files.soft_deleted_at IS NULL)) AND ((((((maven_remote_packages.namespace_id = $3::uuid) AND (maven_remote_packages.maven_remote_repository_id IN ($4::uuid, $5::uuid))) AND (maven_remote_packages.group_id = $6::text)) AND (maven_remote_packages.artifact_id = $7::text)) AND (maven_remote_packages.soft_deleted_at IS NULL)) AND (maven_remote_repositories.namespace_id = $8::uuid));Bound args: [d056e263-dc50-72c1-9612-d24580876b50, maven-metadata.xml, d056e263-dc50-72c1-9612-d24580876b50, e7b6c541-c994-7a28-83dc-9544a4e776c4, a6d6b029-7a4b-75bc-9e8d-a4862fa0d98b, com/example, widget, d056e263-dc50-72c1-9612-d24580876b50]
Plan (EXPLAIN (ANALYZE, BUFFERS) output):
Nested Loop (cost=0.57..21.98 rows=1 width=82) (actual time=0.020..0.026 rows=2 loops=1)
Buffers: shared hit=13
-> Nested Loop (cost=0.57..20.93 rows=1 width=105) (actual time=0.016..0.021 rows=2 loops=1)
Buffers: shared hit=11
-> Index Scan using maven_remote_packages_p41_namespace_id_maven_remote_reposit_idx on maven_remote_packages_p41 maven_remote_packages (cost=0.29..12.61 rows=1 width=48) (actual time=0.009..0.011 rows=2 loops=1)
Index Cond: ((namespace_id = 'd056e263-dc50-72c1-9612-d24580876b50'::uuid) AND (maven_remote_repository_id = ANY ('{e7b6c541-c994-7a28-83dc-9544a4e776c4,a6d6b029-7a4b-75bc-9e8d-a4862fa0d98b}'::uuid[])) AND (group_id = 'com/example'::text) AND (artifact_id = 'widget'::text))
Buffers: shared hit=5
-> Index Scan using maven_remote_files_p41_namespace_id_maven_remote_package_i_idx1 on maven_remote_files_p41 maven_remote_files (cost=0.29..8.31 rows=1 width=105) (actual time=0.004..0.005 rows=1 loops=2)
Index Cond: ((namespace_id = 'd056e263-dc50-72c1-9612-d24580876b50'::uuid) AND (maven_remote_package_id = maven_remote_packages.id))
Filter: ((maven_remote_version_id IS NULL) AND (soft_deleted_at IS NULL) AND (file_name = 'maven-metadata.xml'::text))
Buffers: shared hit=6
-> Seq Scan on maven_remote_repositories_p41 maven_remote_repositories (cost=0.00..1.03 rows=1 width=34) (actual time=0.001..0.001 rows=1 loops=2)
Filter: ((namespace_id = 'd056e263-dc50-72c1-9612-d24580876b50'::uuid) AND (id = maven_remote_packages.maven_remote_repository_id))
Rows Removed by Filter: 0
Buffers: shared hit=2
Planning Time: 2.139 ms
Execution Time: 0.041 msTimings: planning 2.139ms, execution 0.041ms, total 2.180ms.
datastore.ByPositions._VersionNonEmpty_Metadata
Summary: Same shape through the version anchor: the packages index scan feeds the versions index scan (version = $4 applied as a filter on the package's index), which feeds the files index scan on (namespace_id, maven_remote_version_id, file_name). The metadata freshness arm projects NOT snapshot_metadata_always_revalidate AND window — projection only, no filter cost. Pruned to one of 64 partitions on all three tables; 2 actual rows at 0.065ms. No anomalies.
Seed shape: namespaces=1, repositories=2, maven_remote_repositories=2, blob_storage_blobs=1, blob_storage_attachments=1, maven_remote_packages=10002, maven_remote_versions=10002, maven_remote_files=10002 (one version and one version-level metadata file per package)
Rendered SQL:
SELECT maven_remote_packages.maven_remote_repository_id AS "maven_remote_packages.maven_remote_repository_id",
maven_remote_files.blob_sha256 AS "maven_remote_files.blob_sha256",
maven_remote_files.upstream_etag AS "maven_remote_files.upstream_etag",
((NOT maven_remote_repositories.snapshot_metadata_always_revalidate) AND (maven_remote_files.upstream_checked_at > (NOW() - (INTERVAL '1 HOUR' * maven_remote_repositories.metadata_cache_validity_hours)))) AS "fresh"
FROM public.maven_remote_files
INNER JOIN public.maven_remote_versions ON ((maven_remote_versions.id = maven_remote_files.maven_remote_version_id) AND (maven_remote_versions.namespace_id = maven_remote_files.namespace_id))
INNER JOIN public.maven_remote_packages ON ((maven_remote_packages.id = maven_remote_versions.maven_remote_package_id) AND (maven_remote_packages.namespace_id = maven_remote_versions.namespace_id))
INNER JOIN public.maven_remote_repositories ON ((maven_remote_repositories.id = maven_remote_packages.maven_remote_repository_id) AND (maven_remote_repositories.namespace_id = maven_remote_packages.namespace_id))
WHERE (((((((maven_remote_files.namespace_id = $1::uuid) AND (maven_remote_files.maven_remote_version_id IS NOT NULL)) AND (maven_remote_files.file_name = $2::text)) AND (maven_remote_files.soft_deleted_at IS NULL)) AND (maven_remote_versions.namespace_id = $3::uuid)) AND (maven_remote_versions.version = $4::text)) AND (maven_remote_versions.soft_deleted_at IS NULL)) AND ((((((maven_remote_packages.namespace_id = $5::uuid) AND (maven_remote_packages.maven_remote_repository_id IN ($6::uuid, $7::uuid))) AND (maven_remote_packages.group_id = $8::text)) AND (maven_remote_packages.artifact_id = $9::text)) AND (maven_remote_packages.soft_deleted_at IS NULL)) AND (maven_remote_repositories.namespace_id = $10::uuid));Bound args: [8c8b0245-5901-7a4b-bef4-010bef2350c6, maven-metadata.xml, 8c8b0245-5901-7a4b-bef4-010bef2350c6, 2.0.0, 8c8b0245-5901-7a4b-bef4-010bef2350c6, fd2bab8d-e9c9-7645-a3fb-11ef7447a1b8, 53c3dc8c-ef1d-70a4-aed3-4f2226a3f50c, com/example, widget, 8c8b0245-5901-7a4b-bef4-010bef2350c6]
Plan (EXPLAIN (ANALYZE, BUFFERS) output):
Nested Loop (cost=0.86..30.30 rows=1 width=82) (actual time=0.030..0.039 rows=2 loops=1)
Buffers: shared hit=19
-> Nested Loop (cost=0.86..29.24 rows=1 width=105) (actual time=0.026..0.034 rows=2 loops=1)
Buffers: shared hit=17
-> Nested Loop (cost=0.57..20.93 rows=1 width=64) (actual time=0.015..0.020 rows=2 loops=1)
Buffers: shared hit=11
-> Index Scan using maven_remote_packages_p19_namespace_id_maven_remote_reposit_idx on maven_remote_packages_p19 maven_remote_packages (cost=0.29..12.61 rows=1 width=48) (actual time=0.008..0.010 rows=2 loops=1)
Index Cond: ((namespace_id = '8c8b0245-5901-7a4b-bef4-010bef2350c6'::uuid) AND (maven_remote_repository_id = ANY ('{fd2bab8d-e9c9-7645-a3fb-11ef7447a1b8,53c3dc8c-ef1d-70a4-aed3-4f2226a3f50c}'::uuid[])) AND (group_id = 'com/example'::text) AND (artifact_id = 'widget'::text))
Buffers: shared hit=5
-> Index Scan using maven_remote_versions_p19_namespace_id_maven_remote_packag_idx4 on maven_remote_versions_p19 maven_remote_versions (cost=0.29..8.31 rows=1 width=48) (actual time=0.004..0.004 rows=1 loops=2)
Index Cond: ((namespace_id = '8c8b0245-5901-7a4b-bef4-010bef2350c6'::uuid) AND (maven_remote_package_id = maven_remote_packages.id))
Filter: (version = '2.0.0'::text)
Buffers: shared hit=6
-> Index Scan using maven_remote_files_p19_namespace_id_maven_remote_version_id_idx on maven_remote_files_p19 maven_remote_files (cost=0.29..8.31 rows=1 width=105) (actual time=0.007..0.007 rows=1 loops=2)
Index Cond: ((namespace_id = '8c8b0245-5901-7a4b-bef4-010bef2350c6'::uuid) AND (maven_remote_version_id = maven_remote_versions.id) AND (file_name = 'maven-metadata.xml'::text))
Buffers: shared hit=6
-> Seq Scan on maven_remote_repositories_p19 maven_remote_repositories (cost=0.00..1.03 rows=1 width=35) (actual time=0.001..0.001 rows=1 loops=2)
Filter: ((namespace_id = '8c8b0245-5901-7a4b-bef4-010bef2350c6'::uuid) AND (id = maven_remote_packages.maven_remote_repository_id))
Rows Removed by Filter: 0
Buffers: shared hit=2
Planning Time: 1.697 ms
Execution Time: 0.065 msTimings: planning 1.697ms, execution 0.065ms, total 1.762ms.
datastore.ByPositions._VersionNonEmpty_Artifact
Summary: Identical join shape to the metadata arm; the artifact freshness arm adds the zero-pin disjunct (cache_validity_hours = $1) as a projected expression only. The pin constant rides as the statement's first positional argument. Pruned to one of 64 partitions on all three tables; 2 actual rows at 0.058ms. No anomalies.
Seed shape: namespaces=1, repositories=2, maven_remote_repositories=2, blob_storage_blobs=1, blob_storage_attachments=1, maven_remote_packages=10002, maven_remote_versions=10002, maven_remote_files=10002 (one version and one version-level artifact file per package)
Rendered SQL:
SELECT maven_remote_packages.maven_remote_repository_id AS "maven_remote_packages.maven_remote_repository_id",
maven_remote_files.blob_sha256 AS "maven_remote_files.blob_sha256",
maven_remote_files.upstream_etag AS "maven_remote_files.upstream_etag",
((maven_remote_repositories.cache_validity_hours = $1) OR (maven_remote_files.upstream_checked_at > (NOW() - (INTERVAL '1 HOUR' * maven_remote_repositories.cache_validity_hours)))) AS "fresh"
FROM public.maven_remote_files
INNER JOIN public.maven_remote_versions ON ((maven_remote_versions.id = maven_remote_files.maven_remote_version_id) AND (maven_remote_versions.namespace_id = maven_remote_files.namespace_id))
INNER JOIN public.maven_remote_packages ON ((maven_remote_packages.id = maven_remote_versions.maven_remote_package_id) AND (maven_remote_packages.namespace_id = maven_remote_versions.namespace_id))
INNER JOIN public.maven_remote_repositories ON ((maven_remote_repositories.id = maven_remote_packages.maven_remote_repository_id) AND (maven_remote_repositories.namespace_id = maven_remote_packages.namespace_id))
WHERE (((((((maven_remote_files.namespace_id = $2::uuid) AND (maven_remote_files.maven_remote_version_id IS NOT NULL)) AND (maven_remote_files.file_name = $3::text)) AND (maven_remote_files.soft_deleted_at IS NULL)) AND (maven_remote_versions.namespace_id = $4::uuid)) AND (maven_remote_versions.version = $5::text)) AND (maven_remote_versions.soft_deleted_at IS NULL)) AND ((((((maven_remote_packages.namespace_id = $6::uuid) AND (maven_remote_packages.maven_remote_repository_id IN ($7::uuid, $8::uuid))) AND (maven_remote_packages.group_id = $9::text)) AND (maven_remote_packages.artifact_id = $10::text)) AND (maven_remote_packages.soft_deleted_at IS NULL)) AND (maven_remote_repositories.namespace_id = $11::uuid));Bound args: [0, b51f3ad5-1df4-77d9-ac3f-3f283267c8e1, widget-2.0.0.jar, b51f3ad5-1df4-77d9-ac3f-3f283267c8e1, 2.0.0, b51f3ad5-1df4-77d9-ac3f-3f283267c8e1, f8806150-26c7-72fb-b45d-c03e39d95938, 5a0ea06b-1066-7569-9722-3d11f250ca92, com/example, widget, b51f3ad5-1df4-77d9-ac3f-3f283267c8e1]
Plan (EXPLAIN (ANALYZE, BUFFERS) output):
Nested Loop (cost=0.86..30.30 rows=1 width=82) (actual time=0.025..0.033 rows=2 loops=1)
Buffers: shared hit=19
-> Nested Loop (cost=0.86..29.24 rows=1 width=105) (actual time=0.021..0.028 rows=2 loops=1)
Buffers: shared hit=17
-> Nested Loop (cost=0.57..20.93 rows=1 width=64) (actual time=0.016..0.021 rows=2 loops=1)
Buffers: shared hit=11
-> Index Scan using maven_remote_packages_p59_namespace_id_maven_remote_reposit_idx on maven_remote_packages_p59 maven_remote_packages (cost=0.29..12.61 rows=1 width=48) (actual time=0.009..0.011 rows=2 loops=1)
Index Cond: ((namespace_id = 'b51f3ad5-1df4-77d9-ac3f-3f283267c8e1'::uuid) AND (maven_remote_repository_id = ANY ('{f8806150-26c7-72fb-b45d-c03e39d95938,5a0ea06b-1066-7569-9722-3d11f250ca92}'::uuid[])) AND (group_id = 'com/example'::text) AND (artifact_id = 'widget'::text))
Buffers: shared hit=5
-> Index Scan using maven_remote_versions_p59_namespace_id_maven_remote_packag_idx4 on maven_remote_versions_p59 maven_remote_versions (cost=0.29..8.31 rows=1 width=48) (actual time=0.004..0.004 rows=1 loops=2)
Index Cond: ((namespace_id = 'b51f3ad5-1df4-77d9-ac3f-3f283267c8e1'::uuid) AND (maven_remote_package_id = maven_remote_packages.id))
Filter: (version = '2.0.0'::text)
Buffers: shared hit=6
-> Index Scan using maven_remote_files_p59_namespace_id_maven_remote_version_id_idx on maven_remote_files_p59 maven_remote_files (cost=0.29..8.31 rows=1 width=105) (actual time=0.003..0.004 rows=1 loops=2)
Index Cond: ((namespace_id = 'b51f3ad5-1df4-77d9-ac3f-3f283267c8e1'::uuid) AND (maven_remote_version_id = maven_remote_versions.id) AND (file_name = 'widget-2.0.0.jar'::text))
Buffers: shared hit=6
-> Seq Scan on maven_remote_repositories_p59 maven_remote_repositories (cost=0.00..1.03 rows=1 width=34) (actual time=0.001..0.001 rows=1 loops=2)
Filter: ((namespace_id = 'b51f3ad5-1df4-77d9-ac3f-3f283267c8e1'::uuid) AND (id = maven_remote_packages.maven_remote_repository_id))
Rows Removed by Filter: 0
Buffers: shared hit=2
Planning Time: 1.734 ms
Execution Time: 0.058 msTimings: planning 1.734ms, execution 0.058ms, total 1.792ms.