chore(datastore): compose the id-keyed npm remote cache evictions

Why

Step 26 of the S17 Phase 6 plan: the datastore side of npm remote eviction, ahead of the handler arms and the bulk pass (plan Steps 29 and 32). On a remote repository the delete routes evict cached rows rather than deleting owned data (spec: Remote artifact eviction). This MR lands the composers those routes will call: EvictNpmRemotePackage and EvictNpmRemoteVersion set soft_deleted_at on the named row only (the purger reaps the subtree), and DeleteNpmRemoteFile hard-deletes the file row together with its blob_storage_attachments row in one transaction. These are the first delete-side writers over the npm remote cache tables.

What

  • The hosted couplings are absent by design: no packument force-expiry, no rebuild-token rotation, no last-version-removes-package rule, and no size_bytes writes. The spec removes each one for a cache that mirrors its upstream, and size_bytes is the advisory buffered counter AddNpmRemoteVersionSizeBytes documents. The suites pin each absence.
  • npm_leaf_deleter.go gains 2 comment-only lines. The new transaction envelope pairs with NpmLeafDeleter.inReadCommittedTx under the dupl linter, and a one-sided //nolint would leave a fresh finding on the older file. Same companion pattern as a5538719.
  • No plan Status-table row rides this MR. A single table-owner MR fills the Status rows for the whole Phase 6 fan-out, so concurrent step MRs never collide on the same lines.

Diff size

1761 insertions, past the 500-LOC guardrail. Split by file group: composer 423, nolint companion 2, integration suite 1009, unit suite 327. Splitting further would separate the suites from the composer they pin, and the Phase 4 composer MRs took the same one-MR shape.

Test plan

  • go test ./internal/datastore/ (unit: the 13-case guard table, the zero-value contract, and 3 exact-SQL statement pins)
  • go test -tags=integration -count=1 -run 'NpmRemoteEvict' ./internal/datastore/ (testcontainers PG: 4 test functions, all subtests, fixtures seeded through the real cache-fill upserts)
  • golangci-lint at the CI-pinned version, plain and with --build-tags=integration --max-same-issues=0 --max-issues-per-linter=0: no findings in the branch files beyond the package's pre-existing contextcheck seed-helper class
  • gofmt, go vet (plain and integration tags), go build ./...
Spec coverage (from the test-author commit)

Spec coverage

Spec: docs/specs/S17-rest-management-api.md Plan: docs/plans/2026-08-13-s17-phase6-virtual-remote-repositories.md, Step 26

Scope: the internal/datastore composer. Acceptance criteria are numbered by position in the spec's auto-numbered list (123 items); the Phase 6 eviction block spans AC-100 to AC-109.

Acceptance criteria

# Criterion Tests
AC-100 An evicted-but-unreaped row (marked soft_deleted_at) is absent from every list and detail read Composer slice, per the plan (no npm remote management list/detail reads exist yet): invisibility under the suite's own soft-delete predicate (npmRemotePackageRowsByID / npmRemoteVersionRowsByID live counts) and through the marker-filtering point reads (NpmRemotePackageByName, NpmRemoteVersionByPackageAndVersion, NpmRemoteFileByVersionAndName) in TestNpmRemoteEvictor_EvictNpmRemotePackage/marks the named row only and hides it from the package resolve, TestNpmRemoteEvictor_EvictNpmRemoteVersion/marks the named row only and hides it from the version resolve, and TestNpmRemoteEvictor_DeleteNpmRemoteFile/removes the row and its attachment, and touches nothing above. The guard and scope live in the SQL: TestNpmRemoteEvictionMarkStmts
AC-101 DELETE on a remote package, version, and file returns 202 and the target disappears from reads; a missing target returns 404 Handler step (plan Step 29). The composer's share: the disappearance (AC-100 tests above) and the no-op vocabulary the handler's 404 is built on (TestNpmRemoteEvictor_DeleteNpmRemoteFile/an unknown file or version id reports existed=false, the absent-id mark subtests)
AC-102 Evictions emit no artifact_registry_artifact_deleted event Handler step (plan Step 29). The composer has no event surface; its statements reach one table each (TestNpmRemoteEvictionMarkStmts single-table pins)
AC-103 Evicting a remote npm version leaves the cached packument untouched: it serves until its own freshness window expires, and a pull through it re-caches the tarball Composer slice: TestNpmRemoteEvictor_EvictNpmRemoteVersion/leaves the cached packument, the tarball rows, and the advisory size untouched asserts the npm_remote_metadata_files row byte-for-byte (attachment id, digest, etag, upstream_checked_at). The re-cache half is protocol-path behavior outside this step; its datastore substrate (a marked row never conflicts with a fresh fill) is already pinned by TestNpmRemoteVersionStore_UpsertNpmRemoteVersion and TestNpmRemoteFileStore_UpsertNpmRemoteFileForBlob's re-cache-over-tombstone subtests
AC-104 Evicting a remote package removes, once the reap has run, its versions, files, and npm_remote_metadata_files rows, each with its attachment in the same step; a package emptied of versions by eviction remains listed Pre-reap slices and the tail (the reap is S20-A's, per the plan): TestNpmRemoteEvictor_EvictNpmRemotePackage/leaves the subtree and its attachments untouched (marked-but-unreaped package still holds its attachments, children stay live and resolvable) and TestNpmRemoteEvictor_EvictNpmRemoteVersion/a package emptied of versions by eviction remains listed
AC-105 to AC-109 Container eviction, tag upsert 405, remote bulk, remote repository delete Container steps (plan Steps 27, 30, 33) and the bulk and repository-delete steps (Steps 32, and S20-A's plan). No container or route surface here
AC-1 to AC-99, AC-110 to AC-123 Phases 1-4 and 8 surface, virtual repositories, contract documents, and every route-level criterion Not this step. No route, contract, or non-npm surface changes here

Behavior obligations from the spec prose

# Obligation Tests
B-1 Eviction markers: the request sets soft_deleted_at on the NAMED row; row removal is the purger's Mark subtests assert total=1, live=0 (marked, not removed), children live; TestNpmRemoteEvictionMarkStmts pins the single-table UPDATE with the soft_deleted_at IS NULL guard and omits every child table
B-2 Eviction markers: a remote npm file hard-deletes the row and its attachment row, done in the request TestNpmRemoteEvictor_DeleteNpmRemoteFile/removes the row and its attachment, and touches nothing above (attachment gone, CAS blob kept); one-transaction proof in TestNpmRemoteEvictor_TransactionEnvelope (rollback probe brings both rows back; deferred-trigger commit failure reports existed=false with both rows intact)
B-3 No packument force-expiry: the cached document mirrors the upstream and refreshes on its own window The AC-103 subtest (byte-for-byte packument row), plus the same assertion after a file delete in the AC-100 file subtest
B-4 No last-version-removes-package rule: a remote package with no live versions is cold cache TestNpmRemoteEvictor_EvictNpmRemoteVersion/a package emptied of versions by eviction remains listed (package live and resolvable through NpmRemotePackageByName)
B-5 No size adjustment on eviction: size_bytes is the advisory buffered counter, maintained off the request path Non-goal, pinned as untouched: AddNpmRemoteVersionSizeBytes seeds 512 and both the version mark and the file delete leave it at 512; TestNpmRemoteEvictionMarkStmts omits size_bytes
B-6 Idempotent marks: the AND soft_deleted_at IS NULL guard keeps a repeat off the first mark's timestamp .../a repeat call leaves the original mark timestamp in both mark suites (backdated stamp survives the repeat, so a guard-less re-stamping UPDATE fails deterministically); TestNpmRemoteEvictionMarkStmts pins the guard fragment
B-7 Marked subtrees stay reapable: a row under a marked package or version, and a tombstoned row itself, remain hard-deletable so attachments are never stranded TestNpmRemoteEvictor_DeleteNpmRemoteFile/files under a marked subtree stay removable and .../a tombstoned row is still removable, freeing its attachment; TestDeleteNpmRemoteFileStmt pins the absent soft_deleted_at predicate
B-8 Attachment reclaim refuses while another row still references it TestNpmRemoteEvictor_DeleteNpmRemoteFile/an attachment a marked sibling row still references survives (BlobStorageAttachmentStore.DeleteIfUnreferenced composed, marked rows count as references)

Plan obligations (Step 26)

# Obligation Tests
P-1 Exported surface named exactly EvictNpmRemotePackage, EvictNpmRemoteVersion, DeleteNpmRemoteFile in internal/datastore/npm_remote_eviction.go The suites compile against those names; the skeleton fixes the file and signatures
P-2 Marked rows invisible in the composer suite's own queries (the Phase 4 composer pattern) The AC-100 row's own-predicate counts
P-3 File hard-delete removes its attachment row in the same transaction; a marked-but-unreaped package still holds its attachments B-2 and AC-104 rows
P-4 The two no-coupling assertions (packument untouched, emptied package remains) AC-103 and B-4 rows
P-5 Integration walks seeded through the cache upserts seedNpmRemoteEvictionFixture fills through UpsertNpmRemotePackage, UpsertNpmRemoteVersion, UpsertNpmRemoteFileForBlob, and UpsertNpmRemoteMetadataFileForBlob in one committed transaction, with distinct digests asserted (assertDistinctDigests)
P-6 Constructor panics on a nil client; the zero value rejects with a sentinel TestNewNpmRemoteEvictor_NilClientPanics, TestNpmRemoteEvictor_ArgumentGuards (zero-value receiver throughout, nil-client rows)

Error cases

# Condition Tests
E-1 A malformed call: nil context, or a zero namespace, package, version, or file id TestNpmRemoteEvictor_ArgumentGuards (ten guard cases, each on its own sentinel, before any statement runs); a context already done rejects all three methods before any write (TestNpmRemoteEvictor_TransactionEnvelope/a context already done rejects all three methods before any write)
E-2 Artifact delete: target missing or outside the URL's parent chain returns 404 The 404 mapping is the Step 29 handler's, built on the composer's benign no-ops: absent ids, a file addressed through another version, and foreign-namespace ids all write nothing (TestNpmRemoteEvictor_DeleteNpmRemoteFile miss/cross-version/cross-namespace/repeat subtests, the mark suites' miss and tenant subtests)
E-3 The spec's route-level Error Cases rows (400, 401, 403, 409, 413, 422, 503, 405) Route-level; handler and bulk steps. This composer has no request surface

Security considerations

# Concern Tests
S-1 Tenant isolation: every query scoped by namespace_id; a guessed UUID from another namespace resolves nothing Cross-namespace subtests in all three suites (foreign rows untouched and unaddressable); the namespace fragments and bound parameters pinned in TestNpmRemoteEvictionMarkStmts and TestDeleteNpmRemoteFileStmt. The repository-chain check stays with the caller's resolve, pinned as absent by TestNpmRemoteEvictor_DeleteNpmRemoteFile/does not verify the repository chain
S-2 Existence hiding on writes: no failure response states whether a foreign artifact exists A foreign-namespace call and a plain miss return the same no-op vocabulary (nil / existed=false), asserted across the tenant and miss subtests
S-3 Injection: all queries use the Jet builder with bound parameters The composer issues its own SQL: TestNpmRemoteEvictionMarkStmts and TestDeleteNpmRemoteFileStmt assert every scope id is bound as a parameter, never inlined
S-4 Authentication, authorization, delete_artifact on evictions, bounded blast radius Handler and bulk steps behind the S08/S09 stubs (plan Steps 29 and 32). No request surface here

</details>

## Database Review Evidence

### Queries

> [!NOTE]
> Plans are from `EXPLAIN (ANALYZE, BUFFERS)` against an ephemeral
> PostgreSQL 17.10 container (matching `GL_PG_CURR_VERSION` from
> `.gitlab-ci-other-versions.yml`), with synthesized seed data rolled
> back per query and the container torn down at the end of the run.
> Numbers reflect moderate cardinality and do not capture
> production-scale effects. See
> [Database review evidence](docs/dev/database-migrations.md#database-review-evidence)
> for seed sizing, methodology, and the anomalies the skill flags.
> Expand each row's details for the seed shape, rendered SQL, bound args,
> and raw plan.

| Method | Plan node | Index | Rows (plan / actual) | Cost | Time | Buffers (hit / read) | Partitions |
|--------|-----------|-------|----------------------|------|------|----------------------|------------|
| `datastore.DeleteNpmRemoteFile` | Delete | `n/a` | 1 / 1 | 2.88 | 0.015ms | 4 / 0 | 1/64 |
| `datastore.EvictNpmRemotePackage` | Update | `n/a` | 0 / 0 | 1.75 | 0.185ms | 23 / 2 | 1/64 |
| `datastore.EvictNpmRemoteVersion` | Update | `n/a` | 0 / 0 | 1.75 | 0.133ms | 22 / 0 | 1/64 |

<details>
<summary><code>datastore.DeleteNpmRemoteFile</code></summary>

**Summary**: Plan matches the intent of a single-row, id-keyed delete: the bound `namespace_id` prunes to one of 64 partitions, and the root reports 1 / 1 through the RETURNING clause. At the write-target seed (50 rows, two heap pages) the planner correctly prefers a Seq Scan inside the pruned partition, filtering out the 49 sibling rows. Re-seeded at 5000 rows in the same rolled-back session, the planner switches to the partition primary key (`npm_remote_files_p28_pkey`, Index Cond on `id` and `namespace_id`, version id as filter), so the statement is index-served at production cardinality. No anomalies.

**Seed shape**: `namespaces=1, repositories=1, npm_remote_repositories=1, npm_remote_packages=1, npm_remote_versions=1, blob_storage_blobs=50, blob_storage_attachments=50, npm_remote_files=50`

**Rendered SQL**:

```sql
DELETE FROM public.npm_remote_files
WHERE ((npm_remote_files.namespace_id = $1::uuid) AND (npm_remote_files.npm_remote_version_id = $2::uuid)) AND (npm_remote_files.id = $3::uuid)
RETURNING npm_remote_files.blob_storage_attachment_id AS "npm_remote_files.blob_storage_attachment_id",
          npm_remote_files.blob_sha256 AS "npm_remote_files.blob_sha256";

Bound args: ['11111111-1111-1111-1111-111111111111', '55555555-5555-5555-5555-555555555555', '66666666-6666-6666-6666-666666666666']

Plan (EXPLAIN (ANALYZE, BUFFERS) output):

 Delete on npm_remote_files  (cost=0.00..2.88 rows=1 width=10) (actual time=0.013..0.015 rows=1 loops=1)
   Delete on npm_remote_files_p28 npm_remote_files_1
   Buffers: shared hit=4
   ->  Seq Scan on npm_remote_files_p28 npm_remote_files_1  (cost=0.00..2.88 rows=1 width=10) (actual time=0.007..0.008 rows=1 loops=1)
         Filter: ((namespace_id = '11111111-1111-1111-1111-111111111111'::uuid) AND (npm_remote_version_id = '55555555-5555-5555-5555-555555555555'::uuid) AND (id = '66666666-6666-6666-6666-666666666666'::uuid))
         Rows Removed by Filter: 49
         Buffers: shared hit=2
 Planning:
   Buffers: shared hit=351
 Planning Time: 2.085 ms
 Execution Time: 0.170 ms

Timings: planning 2.085ms, execution 0.170ms, total 2.255ms.

datastore.EvictNpmRemotePackage

Summary: Plan matches the intent of a single-row guarded mark: the bound namespace_id prunes to one of 64 partitions, and the scope predicates plus the soft_deleted_at IS NULL idempotence guard discriminate the target from the 49 seeded siblings (scan node 1 / 1). The root's 0 / 0 is the normal shape for DML without RETURNING. At the write-target seed (50 rows, one heap page) the planner correctly prefers a Seq Scan inside the pruned partition. Re-seeded at 5000 rows it switches to the partition primary key (npm_remote_packages_p28_pkey), so the statement is index-served at production cardinality. No anomalies.

Seed shape: namespaces=1, repositories=1, npm_remote_repositories=1, npm_remote_packages=50

Rendered SQL:

UPDATE public.npm_remote_packages
SET soft_deleted_at = NOW()
WHERE ((npm_remote_packages.namespace_id = $1::uuid) AND (npm_remote_packages.id = $2::uuid)) AND (npm_remote_packages.soft_deleted_at IS NULL);

Bound args: ['11111111-1111-1111-1111-111111111111', '44444444-4444-4444-4444-444444444444']

Plan (EXPLAIN (ANALYZE, BUFFERS) output):

 Update on npm_remote_packages  (cost=0.00..1.75 rows=0 width=0) (actual time=0.185..0.185 rows=0 loops=1)
   Update on npm_remote_packages_p28 npm_remote_packages_1
   Buffers: shared hit=23 read=2
   ->  Seq Scan on npm_remote_packages_p28 npm_remote_packages_1  (cost=0.00..1.75 rows=1 width=18) (actual time=0.006..0.008 rows=1 loops=1)
         Filter: ((soft_deleted_at IS NULL) AND (namespace_id = '11111111-1111-1111-1111-111111111111'::uuid) AND (id = '44444444-4444-4444-4444-444444444444'::uuid))
         Rows Removed by Filter: 49
         Buffers: shared hit=1
 Planning:
   Buffers: shared hit=303
 Planning Time: 1.742 ms
 Trigger for constraint fk_npm_remote_packages_namespace_id_namespaces on npm_remote_packages_p28: time=0.360 calls=1
 Trigger for constraint fk_npm_remote_packages_npm_remote_repository_id on npm_remote_packages_p28: time=0.827 calls=1
 Execution Time: 1.538 ms

Timings: planning 1.742ms, execution 1.538ms, total 3.280ms.

datastore.EvictNpmRemoteVersion

Summary: Plan matches the intent of a single-row guarded mark, same shape as the package mark: pruned to one of 64 partitions, target discriminated from the 49 seeded siblings by the scope predicates and the soft_deleted_at IS NULL guard (scan node 1 / 1, root 0 / 0 without RETURNING). At the write-target seed (50 rows, one heap page) the planner correctly prefers a Seq Scan inside the pruned partition. Re-seeded at 5000 rows it switches to the partition primary key (npm_remote_versions_p28_pkey), so the statement is index-served at production cardinality. No anomalies.

Seed shape: namespaces=1, repositories=1, npm_remote_repositories=1, npm_remote_packages=1, npm_remote_versions=50

Rendered SQL:

UPDATE public.npm_remote_versions
SET soft_deleted_at = NOW()
WHERE ((npm_remote_versions.namespace_id = $1::uuid) AND (npm_remote_versions.id = $2::uuid)) AND (npm_remote_versions.soft_deleted_at IS NULL);

Bound args: ['11111111-1111-1111-1111-111111111111', '55555555-5555-5555-5555-555555555555']

Plan (EXPLAIN (ANALYZE, BUFFERS) output):

 Update on npm_remote_versions  (cost=0.00..1.75 rows=0 width=0) (actual time=0.133..0.133 rows=0 loops=1)
   Update on npm_remote_versions_p28 npm_remote_versions_1
   Buffers: shared hit=22 dirtied=2 written=1
   ->  Seq Scan on npm_remote_versions_p28 npm_remote_versions_1  (cost=0.00..1.75 rows=1 width=18) (actual time=0.006..0.008 rows=1 loops=1)
         Filter: ((soft_deleted_at IS NULL) AND (namespace_id = '11111111-1111-1111-1111-111111111111'::uuid) AND (id = '55555555-5555-5555-5555-555555555555'::uuid))
         Rows Removed by Filter: 49
         Buffers: shared hit=1
 Planning:
   Buffers: shared hit=376 read=1
 Planning Time: 1.120 ms
 Trigger for constraint fk_npm_remote_versions_namespace_id_namespaces on npm_remote_versions_p28: time=0.021 calls=1
 Trigger for constraint fk_npm_remote_versions_npm_remote_package_id on npm_remote_versions_p28: time=0.462 calls=1
 Execution Time: 0.778 ms

Timings: planning 1.120ms, execution 0.778ms, total 1.898ms.

Related to #314

Edited by Hayley Swimelar

Merge request reports

Loading
Loading