feat(managementapi): serve the container image delete
Why
S17 Phase 4 gives the management API a delete for every artifact the Phase 3 read surface exposes. This MR is the plan's Step 16 (docs/plans/2026-08-10-s17-phase4-artifact-writes.md): DELETE .../images/{image_id}, the first writer of the container_images soft-delete marker. From the mark onward the image is absent from the management list and detail and from every /v2/ read, the mount source check falls back, and a push reusing the name inserts a fresh row against the partial unique index.
Shipping the first marker writer also makes the manifest-persist re-resolve race reachable: a mark landing between a push's image resolution and the persister's in-transaction re-resolve mints a fresh image row and splits the write, manifest under the fresh row, blobs under the tombstone. persistTx now fails closed when its re-resolve creates a row or finds no live row left, and the OCI push path maps the sentinel to a retryable 503 with Retry-After. This MR is the tracking artifact the persistTx review thread asked for on !1506 (merged).
Stacked on chore(datastore): swap the image name unique in... (!1506 - merged) • Hayley Swimelar • 19.3.
Merge order: merge only after !1506 (merged). The plan records the Step 6 and Step 7 edges as a merge-order requirement rather than a code dependency: this MR is the first marker writer, a mark written before the predicates land is readable, and a re-push before the index swap cannot insert its fresh row. The window closes only when both predecessors are on main.
Deploy order: !1506 (merged) must be fully rolled out, not merely merged, before anything stamps a marker. This MR's own rollout has the same shape one level down: an old pod's push path carries no fail-closed guard, so a marker stamped by a new pod while old pods still serve pushes re-opens the split-write window until the rollout completes.
What (the non-obvious parts)
- Race-fix choice, recorded. The plan text does not allocate the fix mechanism. Failing closed on the re-resolve's created flag is one edit inside the persist transaction (the rollback also removes the fresh row). Threading the resolved image id through the persist input would change the
ociconsumer seam and its adapter, and persisting under the marked row would silently swallow the publish. The sentinel is never retried in process: the rollback removes the fresh row, so every in-process attempt would re-mint and re-fail, which a test pins. - Residual window, tracked. The guard covers both detectable arms: a re-resolve that mints a fresh row (created=true) and one whose conflict read-back finds no live row (the upsert's ErrNotFound, a mark landing between its two statements) both fail closed to the retryable 503. A concurrent push that re-creates the marked name before the re-resolve leaves a created=false row with a different id, and the split persists undetected. Follow-up: Compare the caller-resolved image id in the con... (#624) • Unassigned. The
ErrImageConcurrentlyMarkeddoc records the residual in prose and deliberately names no issue number. - The marker write is repository-scoped in the SQL predicate itself (
namespace_id,container_repository_id,id,soft_deleted_at IS NULL), not only in the caller's prior read. A lost delete race on a live image (the marker write reports no write) answers 202 without an event: only committed deletions emit. oci.error_code=UNAVAILABLEon the push path is client-triggered (a delete raced the push). The response is still a 5xx, so it lands in status-code error-rate signals, the Runway default SLIs included. Excluding it needs a signal keyed on the wide event'soci.error_codeor theerror_codelabel ofgitlab_artifact_registry_oci_manifest_push_failures_total. No shipped SLI does that today.- The 8 added lines in the otherwise-untouched
container_image_integration_test.goare this branch's own lint consequence:duplis symmetric, and the new marker suite's justified//nolint:duplneeds its pair.
Diff size
2608 reviewable LOC: 540 production, 2050 tests, 18 docs. The only clean seam is the persister fail-closed guard, and lifting it into its own MR ships a window where the race is reachable and unguarded. Splitting tests from the code they pin would trade review size for a broken intermediate state on the stack.
Test plan
- Unit: the 404/202/500 handler table over fakes, non-UUID and format-mismatch 404s, repeat-delete 404 with no event, event-after-response ordering via the panicking tracker, the OCI error-taxonomy row for the new sentinel.
- Integration (real PostgreSQL): marker-write idempotency and scoping, gate invisibility at every read, fresh-row re-push, the mark-between-resolves persist race and its no-retry sibling, and the end-to-end walk (push through the protocol store, delete via the handler, management and
/v2/invisibility, re-push, event assertions). - Contract: both
deleteContainerImagearms un-pended, so the sweep now asserts the route serves. - Conformance: the OCI suite exercises live images only and cannot reach the race arm. The unit taxonomy and the mark-race integration suite carry that coverage.
- E2e catalogs: docs/testing/e2e/docker.md and oci.md gain the management-delete journey (delete via the API, pull answers
NAME_UNKNOWN, re-push succeeds as a fresh image).
Context for LLM agents
Design rationale and rejected alternatives:
- Rejected: threading the handler-resolved image id through the persist input in this MR. Footprint (a cross-seam contract change through internal/format/oci and the wire adapter while sibling step branches are in flight), and id-threading alone changes the failure mode to persisting under the marked row, a silently swallowed publish. The follow-up issue adopts id-compare on top of fail-closed for the residual window.
- Rejected: retrying ErrImageConcurrentlyMarked inside the oci store's persist retry loop. The rollback removes the fresh row the re-resolve minted, so each attempt re-mints and re-fails. TestManifestPersister_PersistManifest_MarkRaceIsNotRetried pins this.
- 503 UNAVAILABLE rather than a 4xx or 500: nothing the client sent is wrong, only a full push retry resolves the state, and the code was the one previously-unused member of the OCI error set. Retry-After and the envelope detail carry the retry contract. The value is already in the metrics cardinality closed set.
Non-goals a reviewer might raise:
- No guards in CompleteUpload or MountBlob: their blob writes land under a live row either way, and manifest-push reference validation (scoped to the handler-resolved image id) surfaces residual splits client-visibly.
- The plan's Status-table row for Step 16 stays empty here: a dedicated status-fill wave MR owns those rows in this workstream.
- AC #33 (interrupted reap) and AC #32's removal half are the S20-A purger's behavior. This MR supplies the tombstone its scan discovers.
- Counters do not move on an image delete (AC #49's container arm): versions_count and tags_count are npm_packages columns. The accounting emit is the plan's Step 38.
- Pre-existing spec-step ids ("6b") in internal/format/oci error strings predate this branch and stay.
- Database review evidence (EXPLAIN for the marker UPDATE) is posted to this description as a separate block before reviewer assignment.
Database Review Evidence
Queries
Note
Plans are from EXPLAIN (ANALYZE, BUFFERS) against an ephemeral
PostgreSQL 17 container (matching GL_PG_CURR_VERSION from
.gitlab-ci-other-versions.yml), with synthesized seed data rolled
back per query and the container torn down at the end of the run.
Numbers reflect moderate cardinality and do not capture
production-scale effects. See
Database review evidence
for seed sizing, methodology, and the anomalies the skill flags.
Expand each row's details for the seed shape, rendered SQL, bound args,
and raw plan.
Evidence covers this MR's own DB delta (diffed against
313/step-07-partial-index-swap); the parent's index swap was evidenced
on !1506 (merged). internal/datastore/container_manifest_persister.go also
changed here but adds no new or modified SQL statement (it reads the
created flag the existing UpsertContainerImage upsert already
returned), so the marker UPDATE below is the whole query delta.
| Method | Plan node | Index | Rows (plan / actual) | Cost | Time | Buffers (hit / read) | Partitions |
|---|---|---|---|---|---|---|---|
datastore.SoftDeleteContainerImage |
Update | container_images_p29_pkey |
0 / 0 | 8.31 | 0.228ms | 45 / 0 | 1/64 |
datastore.SoftDeleteContainerImage
Summary: Plan matches the method's intent: a single-row soft delete located by full key. The bound namespace_id prunes at plan time to 1 of 64 hash partitions, and inside it the planner picks the partition primary key (id, namespace_id), applying container_repository_id and soft_deleted_at IS NULL as a filter on the one fetched row (scan rows 1 planned / 1 actual; the root's 0 / 0 is the UPDATE node with no RETURNING). No anomalies.
Seed shape: namespaces=1, repositories=1, container_repositories=1, container_images=5000 (1 live target + 4999 live siblings, all one namespace, so all in one hash partition)
Rendered SQL:
UPDATE public.container_images
SET soft_deleted_at = NOW()
WHERE (((container_images.namespace_id = $1::uuid) AND (container_images.container_repository_id = $2::uuid)) AND (container_images.id = $3::uuid)) AND (container_images.soft_deleted_at IS NULL);Bound args: [6b53cf4b-78bf-4d9d-b64b-9604efd2c946, 5a099fa0-4971-46af-822a-2d35a48df775, 064a7f6f-1086-4c5b-ae02-f242a43bc267] (namespace_id, container_repository_id, id)
Plan (EXPLAIN (ANALYZE, BUFFERS) output):
Update on container_images (cost=0.28..8.31 rows=0 width=0) (actual time=0.228..0.228 rows=0 loops=1)
Update on container_images_p29 container_images_1
Buffers: shared hit=45 dirtied=2 written=1
-> Index Scan using container_images_p29_pkey on container_images_p29 container_images_1 (cost=0.28..8.31 rows=1 width=18) (actual time=0.014..0.014 rows=1 loops=1)
Index Cond: ((id = '064a7f6f-1086-4c5b-ae02-f242a43bc267'::uuid) AND (namespace_id = '6b53cf4b-78bf-4d9d-b64b-9604efd2c946'::uuid))
Filter: ((soft_deleted_at IS NULL) AND (container_repository_id = '5a099fa0-4971-46af-822a-2d35a48df775'::uuid))
Buffers: shared hit=3
Planning:
Buffers: shared hit=149 read=1
Planning Time: 0.630 ms
Trigger for constraint fk_container_images_namespace_id_namespaces on container_images_p29: time=0.022 calls=1
Trigger for constraint fk_container_images_repository_id_container_repositories on container_images_p29: time=2.142 calls=1
Execution Time: 2.559 msTimings: planning 0.630ms, execution 2.559ms, total 3.189ms.
FK-trigger isolation check: the two RI trigger lines (2.164ms of the 2.559ms execution) are a harness artifact, not a statement cost. The UPDATE touches neither FK column, but the rolled-back harness seeds and updates in one transaction, and the RI keys-equal skip does not apply to rows the same transaction inserted. Re-running the identical statement against a committed seed drops the trigger lines entirely:
Update on container_images (cost=0.28..8.31 rows=0 width=0) (actual time=0.419..0.420 rows=0 loops=1)
Update on container_images_p49 container_images_1
Buffers: shared hit=45 dirtied=2 written=1
-> Index Scan using container_images_p49_pkey on container_images_p49 container_images_1 (cost=0.28..8.31 rows=1 width=18) (actual time=0.018..0.019 rows=1 loops=1)
Index Cond: ((id = 'fc57bf35-7873-49eb-817f-20b1a7bae457'::uuid) AND (namespace_id = 'c11b86a7-4192-4833-8a61-a6d88f1396fc'::uuid))
Filter: ((soft_deleted_at IS NULL) AND (container_repository_id = '2afa6ddc-0a41-479e-b300-e25d659f94a1'::uuid))
Buffers: shared hit=3
Planning:
Buffers: shared hit=137 read=1
Planning Time: 1.176 ms
Execution Time: 0.715 ms(The different partition, p49 vs p29, is the different random namespace UUID hashing elsewhere; pruning to one partition holds in both.)
Write-target seed sizing: at the write-target minimum seed (1 target + 49 siblings) the pruned partition is a single page and the planner Seq Scans it on cost (pruning still 1/64; execution 1.428ms). The table row reports the 5000-row seed, where the index path is the planner's own unforced choice, per the seed-sizing rationale in the methodology doc.
Related to #313 (closed)