test(datastore): pin retirement against concurrent shred and rotation

Suleimi Ahmedrequested to merge
sahmed/s04b-step6-retirement-concurrency into main

Implements plan step 6 of docs/plans/2026-08-03-s04b-rotation-tooling.md: forced-interleaving coverage proving RetireKeyVersion serializes against concurrent ShredNamespace and RotateKey through the store's FOR UPDATE barrier.

Four tests extend the existing lock-wait harness (pg_blocking_pids-gated, no sleeps) in internal/datastore/namespace_encryption_keys_concurrency_integration_test.go, covering both interleavings in both commit orders:

  • shred commits first → retirement returns ErrNamespaceShredded; tombstones survive.
  • retirement holds the barrier → the shred provably blocks, then tombstones the survivor.
  • rotation-insert commits first → the resumed retirement re-resolves the active version fresh and still deletes its target (the version that was active at dispatch — the one target choice that makes the fresh post-barrier resolve falsifiable).
  • retirement holds the barrier → the resumed insert lands the next version from MAX(version) over the surviving rows.

Test-only: no production code changes; the store under test merged with !1343 (merged). Deterministic across repeated -race runs.

E2E scenario catalogs

No scenario added or affected: store-level concurrency coverage with no user-facing surface.

Database review

No new migration or query-producing method; the tests drive Step 1's store operations through the existing harness.

Spec coverage

Spec: docs/specs/S04-b-encryption-key-rotation-tooling.md

Acceptance criteria

# Criterion Tests
AC-10 RetireKeyVersion serializes against concurrent ShredNamespace and RotateKey through the shared FOR UPDATE barrier, proven by forced-interleaving integration tests; shred-commits-first yields ErrNamespaceShredded; retirement holding the barrier blocks a concurrent shred until commit TestNamespaceEncryptionKeyStore_RetireVsShred_RetireBlockedResumesShredded, TestNamespaceEncryptionKeyStore_RetireVsShred_ShredBlocksUntilRetireCommits, TestNamespaceEncryptionKeyStore_RetireVsRotate_RetireResumesOnFreshActive, TestNamespaceEncryptionKeyStore_RetireVsRotate_RotateBlocksThenLandsNext

Error cases

# Condition Tests
E-21 Retire: namespace carries any tombstone → ErrNamespaceShredded; tombstones are never deleted Concurrency dimension: TestNamespaceEncryptionKeyStore_RetireVsShred_RetireBlockedResumesShredded (barrier-blocked retirement resumes onto committed tombstones). Committed-state input classes: Step 1's suite (namespace_encryption_keys_rotation_integration_test.go)
E-22 Retire: version at or above the active key → ErrKeyVersionActive Boundary under concurrency: TestNamespaceEncryptionKeyStore_RetireVsRotate_RetireResumesOnFreshActive (the guard resolves post-barrier, so a concurrently-deactivated target is admitted). Refusal itself: Step 1's suite

Security considerations

# Concern Tests
— No security-consideration row names retirement concurrency; the AC-10 barrier serialization above is the relevant control Covered under AC-10

Related to https://gitlab.com/gitlab-org/ops/artifact-registry/-/issues/496

Merge request reports