feat(datastore): add a repository-fenced container image upsert (#1122 part 2 stack: 1/3)

What this delivers

This MR is part 2a of issue 1122, and the first MR of a stack of three:

  1. This MR: the repository-fenced image upsert in the datastore.
  2. The initiate write deadline: !3379 (merged), which targets this branch.
  3. The fence on the OCI write paths: !3368 (merged), which targets the branch of !3379 (merged).

ContainerImageStore.UpsertContainerImageInLiveRepository is a new datastore method. It opens its own READ COMMITTED transaction and calls ReVerifyRepositoryAlive as the first statement. Then it runs the existing UpsertContainerImage in that transaction and commits. If the repository is not live, the method returns ErrRepositoryConcurrentlyDeleted, keeps ErrNotFound out of the error chain, and inserts nothing. The method does not bound the row-lock wait, so the caller supplies the deadline.

Nothing calls the method in this MR. !3368 (merged) switches the OCI image upserter to it. Thus, this MR changes no behavior.

Tests

Test What it asserts
TestContainerImageStore_UpsertContainerImageInLiveRepository_RepositoryGate A live hosted docker or oci repository upserts. A tombstoned repository, a virtual or remote binding, and a Maven-format binding return the sentinel and write no row. A zero container repository id returns a guard error, not the sentinel.
TestContainerImageStore_UpsertContainerImageInLiveRepository_TombstonedMidTransaction The method waits on a tombstone that holds the row lock. When the tombstone commits, the method returns the sentinel and inserts no row.
TestContainerImageStore_UpsertContainerImageInLiveRepository_RolledBackTombstoneUpserts When the tombstone rolls back, the method inserts the row.
TestContainerImageStore_UpsertContainerImageInLiveRepository_TombstoneWaitsForInFlightUpsert The method's INSERT is held in flight after the fence passes, behind an uncommitted row of the same triple. A concurrent tombstone UPDATE waits on the FOR SHARE of the fence, read from pg_stat_activity, until the method commits its row.
TestContainerImageStore_UpsertContainerImageInLiveRepository_RunsAtReadCommitted The transaction runs at READ COMMITTED, also when the session default is REPEATABLE READ.
TestContainerImageStore_UpsertContainerImageInLiveRepository_HonorsCallerDeadline The row-lock wait ends at the deadline of the caller, and the error is the deadline, not the sentinel.
TestContainerImageStore_UpsertInLiveRepository_NilContext A nil context returns errContainerImageNilContext before any database read.

Database review

The method runs three statements in one READ COMMITTED transaction. This MR changes no SQL text: each statement is the same as on main, and only the transaction around them is new.

Order Statement Lock Source
1 ContainerRepositoryStore.ReVerifyRepositoryAlive: SELECT container_repositories.id ... INNER JOIN repositories ... LIMIT 1 FOR SHARE OF repositories FOR SHARE on one repositories row, held to the commit internal/datastore/container_repository.go
2 upsertContainerImageInsert: INSERT INTO container_images (id, namespace_id, container_repository_id, name) VALUES (...) ON CONFLICT (namespace_id, container_repository_id, name) WHERE soft_deleted_at IS NULL DO NOTHING RETURNING ... Row lock on the inserted row internal/datastore/container_image.go
3 findContainerImageByTriple, only when the insert returns no row: SELECT ... FROM container_images WHERE namespace_id = $1 AND container_repository_id = $2 AND name = $3 AND soft_deleted_at IS NULL LIMIT 1 None internal/datastore/container_image.go

No EXPLAIN (ANALYZE, BUFFERS) was captured for this MR. !2268 (merged) records the plan for statement 1 on PostgreSQL 17 with 5,000 seeded repositories: it prunes both tables to 1 of 64 partitions and uses container_repositories_p32_pkey and repositories_p32_pkey, at 1 / 1 rows and 8 shared buffer hits. The statement text has not changed since the !2268 (merged) head 46a5d1121. Statements 2 and 3 bind the full (namespace_id, container_repository_id, name) key of the partial unique index unique_container_images_ns_id_cr_id_name.

The cost of the share lock on the request path, and the k6 runs that measure it, are in !3368 (merged), because that MR adds the callers.

e2e scenario catalog

This MR adds no row and changes no row, because no client route calls the new method yet. !3368 (merged) updates the e2e.oci.setup.delete-repository row.

User documentation

This MR changes no customer-visible surface, so docs/user/ does not change.

Size

520 changed lines (519 added, 1 deleted) in 3 files.

File group Lines changed
Production Go (internal/datastore/container_image.go) 53
Integration tests (internal/datastore/container_image_integration_test.go) 452
Unit tests (internal/datastore/container_image_test.go) 15

The diff is more than 500 lines, but a split does not help the review. The production change is 53 lines, and the tests are the concurrency proof for that change. A split would put the method in one MR and the proof that its lock holds in a different MR.

Merge order

Merge this MR first. !3379 (merged) and !3368 (merged) stack on it, and GitLab retargets each of them when the MR below it merges.

Related to #1122 (closed)

🤖 Generated with Claude Code

Edited by Pawel Rozlach

Merge request reports

Loading
Loading