feat(datastore): add a repository-fenced container image upsert (#1122 part 2 stack: 1/3)
What this delivers
This MR is part 2a of issue 1122, and the first MR of a stack of three:
- This MR: the repository-fenced image upsert in the datastore.
- The initiate write deadline: !3379 (merged), which targets this branch.
- The fence on the OCI write paths: !3368 (merged), which targets the branch of !3379 (merged).
ContainerImageStore.UpsertContainerImageInLiveRepository is a new datastore method.
It opens its own READ COMMITTED transaction and calls ReVerifyRepositoryAlive as the first statement.
Then it runs the existing UpsertContainerImage in that transaction and commits.
If the repository is not live, the method returns ErrRepositoryConcurrentlyDeleted, keeps ErrNotFound out of the error chain, and inserts nothing.
The method does not bound the row-lock wait, so the caller supplies the deadline.
Nothing calls the method in this MR. !3368 (merged) switches the OCI image upserter to it. Thus, this MR changes no behavior.
Tests
| Test | What it asserts |
|---|---|
TestContainerImageStore_UpsertContainerImageInLiveRepository_RepositoryGate |
A live hosted docker or oci repository upserts. A tombstoned repository, a virtual or remote binding, and a Maven-format binding return the sentinel and write no row. A zero container repository id returns a guard error, not the sentinel. |
TestContainerImageStore_UpsertContainerImageInLiveRepository_TombstonedMidTransaction |
The method waits on a tombstone that holds the row lock. When the tombstone commits, the method returns the sentinel and inserts no row. |
TestContainerImageStore_UpsertContainerImageInLiveRepository_RolledBackTombstoneUpserts |
When the tombstone rolls back, the method inserts the row. |
TestContainerImageStore_UpsertContainerImageInLiveRepository_TombstoneWaitsForInFlightUpsert |
The method's INSERT is held in flight after the fence passes, behind an uncommitted row of the same triple. A concurrent tombstone UPDATE waits on the FOR SHARE of the fence, read from pg_stat_activity, until the method commits its row. |
TestContainerImageStore_UpsertContainerImageInLiveRepository_RunsAtReadCommitted |
The transaction runs at READ COMMITTED, also when the session default is REPEATABLE READ. |
TestContainerImageStore_UpsertContainerImageInLiveRepository_HonorsCallerDeadline |
The row-lock wait ends at the deadline of the caller, and the error is the deadline, not the sentinel. |
TestContainerImageStore_UpsertInLiveRepository_NilContext |
A nil context returns errContainerImageNilContext before any database read. |
Database review
The method runs three statements in one READ COMMITTED transaction.
This MR changes no SQL text: each statement is the same as on main, and only the transaction around them is new.
| Order | Statement | Lock | Source |
|---|---|---|---|
| 1 | ContainerRepositoryStore.ReVerifyRepositoryAlive: SELECT container_repositories.id ... INNER JOIN repositories ... LIMIT 1 FOR SHARE OF repositories |
FOR SHARE on one repositories row, held to the commit |
internal/datastore/container_repository.go |
| 2 | upsertContainerImageInsert: INSERT INTO container_images (id, namespace_id, container_repository_id, name) VALUES (...) ON CONFLICT (namespace_id, container_repository_id, name) WHERE soft_deleted_at IS NULL DO NOTHING RETURNING ... |
Row lock on the inserted row | internal/datastore/container_image.go |
| 3 | findContainerImageByTriple, only when the insert returns no row: SELECT ... FROM container_images WHERE namespace_id = $1 AND container_repository_id = $2 AND name = $3 AND soft_deleted_at IS NULL LIMIT 1 |
None | internal/datastore/container_image.go |
No EXPLAIN (ANALYZE, BUFFERS) was captured for this MR.
!2268 (merged) records the plan for statement 1 on PostgreSQL 17 with 5,000 seeded repositories: it prunes both tables to 1 of 64 partitions and uses container_repositories_p32_pkey and repositories_p32_pkey, at 1 / 1 rows and 8 shared buffer hits.
The statement text has not changed since the !2268 (merged) head 46a5d1121.
Statements 2 and 3 bind the full (namespace_id, container_repository_id, name) key of the partial unique index unique_container_images_ns_id_cr_id_name.
The cost of the share lock on the request path, and the k6 runs that measure it, are in !3368 (merged), because that MR adds the callers.
e2e scenario catalog
This MR adds no row and changes no row, because no client route calls the new method yet.
!3368 (merged) updates the e2e.oci.setup.delete-repository row.
User documentation
This MR changes no customer-visible surface, so docs/user/ does not change.
Size
520 changed lines (519 added, 1 deleted) in 3 files.
| File group | Lines changed |
|---|---|
Production Go (internal/datastore/container_image.go) |
53 |
Integration tests (internal/datastore/container_image_integration_test.go) |
452 |
Unit tests (internal/datastore/container_image_test.go) |
15 |
The diff is more than 500 lines, but a split does not help the review. The production change is 53 lines, and the tests are the concurrency proof for that change. A split would put the method in one MR and the proof that its lock holds in a different MR.
Merge order
Merge this MR first. !3379 (merged) and !3368 (merged) stack on it, and GitLab retargets each of them when the MR below it merges.
Related to #1122 (closed)