Look up tokens and digests with every db_key_base key

Rémy Coutablerequested to merge
631457-db-key-base-tokens-digests into master

What does this MR do and why?

Phase 2 of making db_key_base rotatable: tokens, digests and signatures derived from db_key_base keep working under every configured key.

db_key_base can be an array: the last key is current, every key decrypts. During a rotation ([new, old], then [old, new]), lookups that compute a value with the current key only and compare it to what is stored would fail. That affects PAT login, runner, deploy, trigger, agent, build and SCIM token lookups, secret detection lookups, webhook log URL checks, JWT, LFS and dependency proxy tokens, and Geo logout state.

Nothing changes for instances today: config/settings.rb still raises when more than one key is configured (Phase 3 removes that). With a single key, every digest, ciphertext and signing secret is byte-identical to today.

Changes, by area:

  • Candidates
    • Gitlab::CryptoHelper.sha256_candidates and aes256_gcm_encrypt_candidates return one value per key, current key first.
    • encryption_key is no longer memoized in the module (KeyProvider already memoizes).
  • Encrypted token lookups use every key: Authn::TokenField::EncryptionHelper.encrypt_token_candidates, encode_candidates on the encrypted and digest strategies and on TokenAuthenticatable models, Authn::TokenField::Finders::BaseEncrypted (also behind the partitioned job and runner finders), Ci::Trigger.with_token, Ci::Runner.search, WebHookLog#url_current?, and EE Security::SecretDetection::TokenLookupService (falls back to encode for models without candidates, such as OAuth applications, whose secrets do not depend on db_key_base).
  • Digest tokens (personal access tokens, EE runner controller tokens) are found under any key's salt. Rewriting them with the current salt on use is in !259034.
  • JWT/HMAC verification accepts every key: Gitlab::JWTToken.decode, the dependency proxy token (Auth::ContainerProxyAuthenticationService.secrets), Gitlab::LfsToken#token_valid? and EE Gitlab::Geo::Oauth::LogoutState#decode. New tokens are still signed with the current key.

Out of scope: tokens found under an old key are not re-encrypted here; the Phase 4 rake task does that. Not changed because they are short-lived or never compared: rate-limit scopes, the GitHub import cache key, release evidence summary_sha, WebHookLog#url_hash writes. Stored digests that can't be recomputed (credit card hashes, analytics ids, import source identifiers) still use the current key, as on master; the approach is tracked in https://gitlab.com/gitlab-org/gitlab/-/work_items/631665.

Specs cover each lookup and verifier with a token created under the previous key. A mutation check (candidates returning only the current key) makes 9 of the new lookup examples fail. This adds the spec helper stub_db_key_base_keys(*keys), the same as in !258816 (merged).

References

Screenshots or screen recordings

Not applicable, backend only.

How to set up and validate locally

  1. Run the specs:

    bin/rspec spec/lib/gitlab/crypto_helper_spec.rb spec/lib/authn/token_field spec/lib/gitlab/jwt_token_spec.rb spec/lib/gitlab/lfs_token_spec.rb spec/services/dependency_proxy/auth_token_service_spec.rb spec/models/ci/trigger_spec.rb spec/models/hooks/web_hook_log_spec.rb ee/spec/lib/gitlab/geo/oauth/logout_state_spec.rb ee/spec/services/security/secret_detection/token_lookup_service_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Rémy Coutable

Merge request reports