feat(crypto): add NamespaceKeyManager with TTL cache, auto-create, and crypto-shredding

What

Adds DefaultNamespaceKeyManager (internal/crypto): the tier-2 namespace-KEK cache and lifecycle layer over the Step 3 store — a TTL-bounded in-memory cache of unwrapped namespace keys, auto-create on first use, version lookups, and crypto-shredding. Step 4 of the S04-A column-level-encryption plan.

Contract highlights

• Copy-on-serve: every returned NamespaceKey is an independent, caller-owned copy; the manager never aliases its cache, so a consumer that zeroizes a served key cannot corrupt a cached key or another caller's copy. Now stated on the NamespaceKeyManager interface so implementers are bound.
• Fallback-unwrap hook scoped to GetKey; partial-plaintext zeroization on unwrap/insert error; version-lookup caching.

Plan amendments carried by this MR

Two amendments to the merged S04-A plan ship in this MR:

Commit 4e8d493f (docs/plans) widens the approved Step-4 scope line with the constructor-started background cache sweep (and its cleanup-function return shape), the resolution of the residence-bound discussion tracked in #409 (closed).

Commit 5d59e164 (docs/plans) adds a Step-5 contract line: EncryptRow acquires the namespace key itself and takes no caller-supplied key. The rationale is this step's copy-on-serve design (a served NamespaceKey is a caller-owned copy with no TTL), so the amendment ships here rather than in the Step-5 MR.

Stacking

Stacked on !964 (merged) (Step 3), targets main. Until !964 (merged) lands the diff shows Step 3's changes too; once !964 (merged) merges only this branch's changes remain.

Spec coverage

Spec: docs/specs/S04-a-column-level-encryption.md

Scope: plan Step 4, DefaultNamespaceKeyManager only — GetKey, GetKeyByVersion, GetKeyByID, RotateKey (rotation; version-1 auto-create goes through the store's InsertInitialKey), Shred, the three cache maps (active, version, id), the constructor-started expiry sweeper, the expiry anchor, and the copy-on-serve contract. Primitives (Step 1), LocalProvider (Step 2), the NamespaceKeyStore SQL (Step 3), RowEncryptor (Step 5), rotation drivers (Step 6), and RedactedString (Step 7) are out of scope and are not listed. Manager tests are split per read-path family: namespace_key_active_test.go (GetKey), namespace_key_version_test.go, namespace_key_id_test.go, namespace_key_rotate_test.go, namespace_key_shred_test.go, namespace_key_sweep_test.go (sweeper), namespace_key_initial_test.go (auto-create/adoption), with shared fixtures, constructor, and cross-family tests in namespace_key_test.go; plus the datastore integration suites (namespace_encryption_keys_integration_test.go incl. the GetKeyByID cases, ..._adoption_integration_test.go, ..._concurrency_integration_test.go, ..._initial_integration_test.go where present).

Acceptance criteria

#	Criterion	Tests
AC-8	GetKey returns a valid key and auto-creates for a new namespace	TestManager_GetKey_AutoCreatesForNewNamespace, TestManager_GetKey_UnwrapsExistingActiveKey, TestManager_UnwrapForwardsStoreRecordVerbatim, TestManager_GetKey_ServesKeyRowID
AC-9	GetKeyByVersion retrieves deactivated keys	TestManager_GetKeyByVersion_ReturnsDeactivatedKey, TestManager_GetKeyByVersion_ServesKeyRowID, TestManager_GetKey_PopulatesVersionEntry
AC-12	Root key rotation: old namespace keys still unwrap	Manager half only (a record wrapped under a retained root key unwraps, and signals): TestManager_GetKey_FallbackUnwrapHook, TestManager_GetKey_NoFallbackHookOnActiveKey, TestManager_GetKeyByVersion_DoesNotFireFallbackHook. The RotateAll sweep and "new writes use the new root key" are Step 6; not in this MR.
AC-13	Namespace key rotation: old versions still resolve, new writes use the new version	Manager half only: TestManager_RotateKey_RotatesActiveVersion, TestManager_RotateKey_RotationZeroizesReplacedActiveEntry, TestManager_RotateKey_ServesKeyRowID, TestManager_GetKeyByVersion_MirroredEntrySurvivesRotation, TestManager_RotateKey_RotationWinsAgainstAnEquallyDatedEntry. Credential-row re-wrapping is Step 5/6.
AC-15	Crypto-shredding: key material unrecoverable after shred	Manager half only (cache eviction, plaintext zeroization, fail-closed): TestManager_Shred_EvictsAndZeroizesThenFailsClosed, TestManager_Shred_EvictsVersionEntries, TestManager_Shred_DoesNotEvictOtherNamespaces, TestManager_Shred_StoreFailureKeepsCache. Durable row destruction is store-owned (Step 3).
AC-17	Cache respects TTL; Shred evicts and zeroizes cached keys	TestManager_GetKey_CacheHitAndTTLExpiry, TestManager_GetKeyByVersion_CacheHitAndTTLExpiry, TestManager_GetKey_CacheEntryExpiresRelativeToTheRead, TestManager_GetKeyByVersion_CacheEntryExpiresRelativeToTheRead, TestManager_GetKey_MirroredVersionEntryExpiresRelativeToTheRead, TestManager_RotateKey_CacheEntryExpiresRelativeToTheInsert, TestManager_GetKey_ConflictReReadAnchorsExpiryToItsOwnRead, TestManager_PutVersion_OneInstantJudgesLiveEntryAndCandidate, TestManager_PutID_OneInstantJudgesLiveEntryAndCandidate, TestManager_GetKey_MirrorZeroizesReplacedVersionEntry, TestManager_Shred_EvictsAndZeroizesThenFailsClosed, TestManager_Shred_EvictsVersionEntries
AC-18	Post-shred GetKey returns ErrNamespaceShredded durably via the marker; no auto-create; survives restart; holds fleet-wide	TestManager_GetKey_ShreddedFailsClosed, TestManager_GetKeyByVersion_ShreddedFailsClosed, TestManager_GetKey_ShreddedDurableAcrossManagers (a second manager over the same store models restart / another fleet instance)
AC-21	./internal/crypto/... passes under GODEBUG=fips140=only	CI gate, not a manager test. Enforced by the .gitlab-ci.yml FIPS job over the whole package; the assertion helper's tests live in the pre-existing internal/crypto/fips_test.go, unchanged by this MR.
AC-22	All tests pass with -race	CI gate. Manager-specific race exercise: TestManager_GetKey_ConcurrentReadsConsistent, TestManager_GetKey_ConcurrentWithShred.

Error cases

#	Condition	Tests
E-5	Concurrent first-write auto-create race: loser gets ErrKeyConflict and re-reads GetActiveKey	TestManager_GetKey_ReReadsOnKeyConflict, TestManager_GetKey_ConflictThenShreddedFailsClosed, TestManager_GetKey_ConflictThenNoRowFailsWithoutRetry, TestManager_GetKey_ConflictReReadAnchorsExpiryToItsOwnRead (the 23505 to ErrKeyConflict mapping is store-owned)
E-6	Namespace has no keys and no marker: GetKey auto-creates one	TestManager_GetKey_AutoCreatesForNewNamespace
E-7	Key mutation for a namespace with no namespaces parent row returns ErrNamespaceNotFound	Propagation only: TestManager_GetKey_AutoCreateNamespaceNotFound, TestManager_RotateKey_NamespaceNotFound. The 23503 mapping is store-owned (Step 3).
E-8	Key version not found	TestManager_GetKeyByVersion_NotFound (the DecryptRow variant is Step 5)
E-14	RootKeyProvider unavailable: operation returns the error, caches nothing, leaks no plaintext	TestManager_GetKey_ProviderUnwrapError, TestManager_GetKey_ZeroizesPartialPlaintextOnUnwrapError, TestManager_RotateKey_ZeroizesPlaintextOnWrapError, TestManager_RotateKey_ZeroizesPlaintextOnInsertError
E-15	Shredded namespace, read path: ErrNamespaceShredded, durable, no auto-create	TestManager_GetKey_ShreddedFailsClosed, TestManager_GetKeyByVersion_ShreddedFailsClosed, TestManager_GetKey_ShreddedDurableAcrossManagers, TestManager_GetKey_ConflictThenShreddedFailsClosed
E-16	Shredded namespace, write path: RotateKey and InsertInitialKey return ErrNamespaceShredded, no resurrection	TestManager_RotateKey_ShreddedIsRefused, TestManager_Shred_EvictsAndZeroizesThenFailsClosed
E-21	Lookup or insert stalled longer than the cache TTL: zeroize, cache nothing, return ErrKeyReadExpired, unless a live cached entry supersedes it; RotateKey's insert still stands	TestManager_GetKey_ReadOlderThanTTLFailsClosed, TestManager_GetKeyByVersion_ReadOlderThanTTLFailsClosed, TestManager_RotateKey_InsertOlderThanTTLFailsClosedWithTheRowCommitted, TestManager_GetKey_AutoCreateOlderThanTTLFailsClosedWithoutRetry, TestManager_GetKey_ExpiredPublicationYieldsToTheLiveEntry, TestManager_GetKeyByVersion_RefusedPublicationServesTheLiveWinner, TestManager_GetKey_RefusedPublishServesIndependentCopy, TestManager_GetKey_StalledLookupCannotDowngradeRotatedVersion, TestManager_GetKey_StalledLookupCannotShortenAnEqualVersionEntry, TestManager_GetKeyByVersion_StalledPublishRefusedByLaterMirrorIsWiped, TestManager_RotateKey_ExpiredHigherVersionDoesNotBlockReEnable

Security considerations

#	Concern	Tests
S-1	Key plaintext zeroized after use (best-effort)	Manager-owned KEK zeroization: TestManager_Shred_EvictsAndZeroizesThenFailsClosed, TestManager_RotateKey_RotationZeroizesReplacedActiveEntry, TestManager_GetKey_MirrorZeroizesReplacedVersionEntry, TestManager_GetKey_ZeroizesPartialPlaintextOnUnwrapError, TestManager_RotateKey_ZeroizesPlaintextOnWrapError, TestManager_RotateKey_ZeroizesPlaintextOnInsertError, TestManager_GetKeyByVersion_StalledPublishRefusedByLaterMirrorIsWiped. Per-row DEK zeroization is Step 5.
S-2	Namespace KEKs cached with a TTL (bounded in-memory exposure)	TestManager_GetKey_CacheHitAndTTLExpiry, TestManager_GetKeyByVersion_CacheHitAndTTLExpiry, plus the AC-17 anchor tests. Construction guards that make the bound meaningful: TestNewDefaultNamespaceKeyManager_NonPositiveTTLPanics, TestNewDefaultNamespaceKeyManager_NilDependencyPanics. The "root KEK resident for process lifetime" half is LocalProvider-owned (Step 2).
S-5	No key material or namespace identifiers escape the package	TestManager_GetKey_FallbackUnwrapHook pins the hook's arguments to two root-key URIs with no namespace ID. Package crypto has no logger and no metrics, so there is no test asserting the absence of logging — the property is structural, not asserted. RedactedString is Step 7.
S-7	Cross-namespace isolation at the cache layer	TestManager_Shred_DoesNotEvictOtherNamespaces. Cryptographic cross-namespace isolation (AC-11) is Step 5.
S-12	Copy-on-serve: a served key is caller-owned; zeroizing it cannot corrupt the cache or another caller	TestManager_GetKey_CacheHitServesIndependentCopy, TestManager_GetKeyByVersion_CacheHitServesIndependentCopy, TestManager_GetKey_ServedKeyIndependentOfShred, TestManager_ServedKeysIndependentOfShred_VersionAndCreate, TestManager_GetKey_RefusedPublishServesIndependentCopy, TestManager_CacheHitServesKeyRowID, TestCopyKey_CarriesKeyRowID. (Not a numbered spec bullet; added because this MR introduces the contract in provider.go.)
S-13	UnwrapKey output length is not validated before caching	No test — behavior not implemented. The spec scopes validation to the first KMS-backed provider; under the in-process LocalProvider a wrong-length KEK still fails closed downstream via ErrKeySizeMismatch. Recorded so the gap is visible, not hidden.
S-14	Concurrent cache misses are not coalesced	No test — behavior deliberately absent. Deferred to the first KMS provider per the plan's out-of-scope note; under the in-process provider a duplicate unwrap is a local decrypt and each displaced entry is still zeroized on replacement.

Guardrails

• Tests: unit coverage for cache TTL, auto-create, shred eviction, copy-before-publish, fail-closed re-read, and mutation-pinned copy/zeroize invariant paths.
• e2e catalog (docs/testing/): not affected — internal crypto plumbing, not yet wired to a user-facing endpoint (wiring is a later S04-A step), so no end-to-end scenario changes.
• Configuration reference: no config schema/loader change in this MR.

Related to https://gitlab.com/gitlab-org/ops/artifact-registry/-/issues/384