feat(npm): any-kind resolution and remote wiring (S15 Step 8, part 2/2)

David Fernandezrequested to merge
10io/s15-step-8-part2of2 into main

Split into 2 stacked MRs to keep each part within the review size limit (the full step is ~2,400 changed lines). Part 1 has merged, so this part now targets main directly.

Stacked MRs (review/merge bottom-up)

Part 2 of 2 of docs/plans/2026-07-15-npm-remote.md — Step 8: kind-dispatch seam + write-unavailability. Spec: docs/specs/S15-npm-remote.md.

🎯 What this part does

Turns resolution on for the kind-dispatch contract landed in part 1:

  • datastore: FindByNameInNamespace resolves an npm repository of any kind and projects repositories.kind through the same JOIN (format, soft-delete, and namespace gates unchanged). FindHostedByNameInNamespace survives as a thin hosted-only wrapper for the management API's NpmRepositoryResolver seam — any other kind reads as ErrNotFound, the same existence-hiding the old SQL predicate enforced.
  • resolver: three-finder NewResolver; a kind=2 resolution attaches the npm_remote_repositories row (upstream URL, self-redacting token, cache-validity windows). A missing remote binding reads as repository_not_found (existence-hiding). The remote tier is keyed by the repositories.id the previous tier resolved, not by the name it resolved it from, so the two queries cannot describe different repositories and no cross-tier identity check is needed. One guard remains, against a misbehaving finder: a (nil, nil) return, which the id key says nothing about.
  • composition root: the remote.AuditSink the credential seam renders through, over structured logging (a pre-actor structured audit payload, not yet a conforming S03-A wide event — no typed emitter, no operation/outcome/duration_s, no actor; registered with its deviations in docs/dev/observability.md, and no credential material either way); the remote finder adapter threaded into the npm resolver; a startup breadcrumb when either threshold would silently suppress the Info-level credential-write audit trail — log.level for writes inside a request, GITLAB_LOG_LEVEL for writes outside one.
  • integration: the composed boot probe drives a kind=2 publish PUT to its 405 and a packument GET to the interim 501 over HTTP through the production store scan, wire adapters, resolver, and dispatcher — the one test that exercises every link of the chain at once.

🔍 Contract coverage

Contract Test
Every repositories.kind value resolves as a positive hit and reports its kind TestNpmRepositoryStore_FindByNameInNamespace
Both name-keyed finders reject caller bugs as guard sentinels, never as a miss TestNpmRepositoryStore_NameKeyedFinders_ArgumentGuards
Hosted-only wrapper hides non-hosted kinds as ErrNotFound TestNpmRepositoryStore_FindHostedByNameInNamespace
kind=2 resolution attaches the remote row; virtual never calls the remote finder TestResolver_Resolve_RemoteKindAttachesRemoteRow, _VirtualKindResolvesWithoutRemoteRow
Missing remote binding reads as repository-not-found, and is recorded at Info TestResolver_Resolve_RemoteBindingMissing, _RemoteUnderlyingErrorDoesNotWarn
Finder-contract breaches fail as internal errors, with an attributable error_code TestResolver_Resolve_RemoteNilWithoutError, TestMiddleware_UnderlyingError_LogsAttributableErrorCode
Production chain end-to-end: remote write 405, remote read 501 over HTTP TestWireNPM_BootsHandlersAgainstRealDB (remote-kind subtests)
Adapter error translation for the remote finder TestNpmRemoteRepositoryFinderAdapter_TranslatesErrors
Audit payload at Info, dropped by a higher threshold TestRemoteAuditLogSink_EmitsStructuredPayload, _DropsAboveItsLevel, _NoContextLoggerHonoursEnvLevel
Boot breadcrumb when either threshold would discard the audit trail TestLogCredentialAuditSuppression, _OffRequestFallback
Remote tier is keyed by the id the previous tier resolved TestNpmRemoteRepositoryStore_FindRemoteByRepositoryID, TestResolver_Resolve_RemoteKindAttachesRemoteRow

📏 Size

~2,100 changed lines across 35 files (grown by the review rounds on this branch), over the stack's usual per-part limit; operator-approved as the compile-atomic remainder (the cmd/ wiring, the three-finder resolver, and the any-kind finder cannot land separately without a broken intermediate state).

📝 Plan amendment approval

The operator approved amending docs/plans/2026-07-15-npm-remote.md in band with this MR rather than through a separate plan MR first. The approval covers the accepted-smells block this MR adds and the token-lifetime clause added to it in f1deeff1. It approves the in-band sequencing only, not the plan's content.

📖 E2E scenario catalogs

No docs/testing/ update: no npm e2e catalog exists yet (docs/testing/e2e/ holds docker and oci only). The remote-read 501s are interim, and the 405 write-unavailability contract lands in the npm-remote catalog together with the proxy handlers.

Database Review Evidence

Queries

Note

Plans are from EXPLAIN (ANALYZE, BUFFERS) against an ephemeral PostgreSQL 17 container (matching GL_PG_CURR_VERSION from .gitlab-ci-other-versions.yml), with synthesized seed data rolled back per query and the container torn down at the end of the run. Numbers reflect moderate cardinality and do not capture production-scale effects. See Database review evidence for seed sizing, methodology, and the anomalies the skill flags. Expand each row's details for the seed shape, rendered SQL, bound args, and raw plan.

Method Plan node Index Rows (plan / actual) Cost Time Buffers (hit / read) Partitions
datastore.NpmRepositoryStore.FindByNameInNamespace Limit repositories_..._namespace_id_format_name_idx, npm_repositories_..._namespace_id_repository_id_idx 1 / 1 16.62 0.026ms 9 / 0 1 of 64 (each table)
datastore.NpmRepositoryStore.FindHostedByNameInNamespace (pre-change form, comparison baseline) Limit same two indexes 1 / 1 16.62 0.026ms 9 / 0 1 of 64 (each table)

Query notes:

  • NpmRepositoryStore.FindHostedByNameInNamespace (as it exists on this branch) is a Go-level wrapper over FindByNameInNamespace: it issues the same single SQL statement and adds no SQL of its own — the kind = hosted narrowing happens in Go on the projected repositories.kind value. Its database behavior is fully covered by the FindByNameInNamespace row above.
  • The pre-change (merge-base) form of the query is included as a comparison baseline only; it is not a query this MR ships.
  • Verdict: the plan is unchanged relative to the old kind-predicated query — same Nested Loop over the same two index scans ((namespace_id, format, name) partial index on repositories, unique (namespace_id, repository_id) on npm_repositories) with identical single-partition pruning and cost; the only differences are the disappearance of the residual Filter: (kind = 0) (which was never an index condition) and a 2-byte-wider output row from projecting repositories.kind.
datastore.NpmRepositoryStore.FindByNameInNamespace

Summary: Plan matches the method's intent: plan-time partition pruning to a single repositories partition and a single npm_repositories partition via the namespace_id literal, Index Scan over the composite (namespace_id, format, name) partial index (WHERE soft_deleted_at IS NULL), then a Nested Loop into the npm binding row via the unique (namespace_id, repository_id) index. Estimates match reality (1 / 1) and execution stays under 50us at 5000 seeded rows per table. No anomalies.

Seed shape: namespaces=1, repositories=5000, npm_repositories=5000 (all rows in one namespace, so both partitioned read targets land in a single partition; kind cycles 0/1/2 across the seeded repositories so the any-kind query runs against mixed kinds)

Rendered SQL:

SELECT npm_repositories.id AS "npm_repositories.id",
     npm_repositories.repository_id AS "npm_repositories.repository_id",
     repositories.kind AS "repositories.kind"
FROM public.npm_repositories
     INNER JOIN public.repositories ON ((repositories.id = npm_repositories.repository_id) AND (repositories.namespace_id = npm_repositories.namespace_id))
WHERE (((npm_repositories.namespace_id = $1::uuid) AND (repositories.name = $2::text)) AND (repositories.format = $3)) AND (repositories.soft_deleted_at IS NULL)
LIMIT $4;

Bound args: [ba8a0a61-c497-4913-84c5-4fca3444df86, 'review-prep-repositories-name-000300', 2, 1]

Plan (EXPLAIN (ANALYZE, BUFFERS) output):

 Limit  (cost=0.56..16.62 rows=1 width=34) (actual time=0.025..0.026 rows=1 loops=1)
   Buffers: shared hit=9
   ->  Nested Loop  (cost=0.56..16.62 rows=1 width=34) (actual time=0.025..0.025 rows=1 loops=1)
         Buffers: shared hit=9
         ->  Index Scan using repositories_p07_namespace_id_format_name_idx on repositories_p07 repositories  (cost=0.28..8.30 rows=1 width=34) (actual time=0.018..0.018 rows=1 loops=1)
               Index Cond: ((namespace_id = 'ba8a0a61-c497-4913-84c5-4fca3444df86'::uuid) AND (format = 2) AND (name = 'review-prep-repositories-name-000300'::text))
               Buffers: shared hit=6
         ->  Index Scan using npm_repositories_p07_namespace_id_repository_id_idx on npm_repositories_p07 npm_repositories  (cost=0.28..8.30 rows=1 width=48) (actual time=0.006..0.006 rows=1 loops=1)
               Index Cond: ((namespace_id = 'ba8a0a61-c497-4913-84c5-4fca3444df86'::uuid) AND (repository_id = repositories.id))
               Buffers: shared hit=3
 Planning:
   Buffers: shared hit=720 read=1
 Planning Time: 1.777 ms
 Execution Time: 0.042 ms

Timings: planning 1.777ms, execution 0.042ms, total 1.819ms.

datastore.NpmRepositoryStore.FindHostedByNameInNamespace (pre-change form, comparison baseline)

Summary: The merge-base form of the finder — kind = hosted predicate present, no kind projection — rendered and explained against an identical seed for comparison. The plan is the same shape node-for-node: same two index scans, same Nested Loop, same single-partition pruning, same cost (16.62). The dropped predicate was never an index condition; it applied as a residual Filter: (kind = 0) on the repositories index scan, so removing it changes nothing about index choice or pruning. The added repositories.kind projection only widens the output row (width 34 vs 32).

Seed shape: namespaces=1, repositories=5000, npm_repositories=5000 (identical seed plan to the row above; the different partition suffix in the plan reflects the per-run random namespace UUID hashing to a different partition, not a pruning difference)

Rendered SQL:

SELECT npm_repositories.id AS "npm_repositories.id",
     npm_repositories.repository_id AS "npm_repositories.repository_id"
FROM public.npm_repositories
     INNER JOIN public.repositories ON ((repositories.id = npm_repositories.repository_id) AND (repositories.namespace_id = npm_repositories.namespace_id))
WHERE ((((npm_repositories.namespace_id = $1::uuid) AND (repositories.name = $2::text)) AND (repositories.format = $3)) AND (repositories.kind = $4)) AND (repositories.soft_deleted_at IS NULL)
LIMIT $5;

Bound args: [442270e3-3cde-4833-b1de-b3948129ac10, 'review-prep-repositories-name-000300', 2, 0, 1]

Plan (EXPLAIN (ANALYZE, BUFFERS) output):

 Limit  (cost=0.56..16.62 rows=1 width=32) (actual time=0.026..0.026 rows=1 loops=1)
   Buffers: shared hit=9
   ->  Nested Loop  (cost=0.56..16.62 rows=1 width=32) (actual time=0.025..0.025 rows=1 loops=1)
         Buffers: shared hit=9
         ->  Index Scan using repositories_p18_namespace_id_format_name_idx on repositories_p18 repositories  (cost=0.28..8.31 rows=1 width=32) (actual time=0.018..0.019 rows=1 loops=1)
               Index Cond: ((namespace_id = '442270e3-3cde-4833-b1de-b3948129ac10'::uuid) AND (format = 2) AND (name = 'review-prep-repositories-name-000300'::text))
               Filter: (kind = 0)
               Buffers: shared hit=6
         ->  Index Scan using npm_repositories_p18_namespace_id_repository_id_idx on npm_repositories_p18 npm_repositories  (cost=0.28..8.30 rows=1 width=48) (actual time=0.005..0.005 rows=1 loops=1)
               Index Cond: ((namespace_id = '442270e3-3cde-4833-b1de-b3948129ac10'::uuid) AND (repository_id = repositories.id))
               Buffers: shared hit=3
 Planning:
   Buffers: shared hit=321
 Planning Time: 0.926 ms
 Execution Time: 0.038 ms

Timings: planning 0.926ms, execution 0.038ms, total 0.964ms.

Related to #345 (closed)

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports