## Summary This epic tracks the implementation of the "Encryption keys rotation" proposal at https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/encryption_keys_rotation/. The goal is to replace our current usage of `attr_encrypted` and `TokenAuthenticatable` (with custom encryption strategy) with Rails's `ActiveRecord::Encryption` framework, and to implement an automated process running in the background that would re-encrypt data upon key rotation. This will allow moving organizations from one cell to another (or to Dedicated). Lastly, a new admin UI will be provided to keep track of encryption keys usage. ## Objectives See the [Objectives of the proposal](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/encryption_keys_rotation/#objectives). ### Why switching to `ActiveRecord::Encryption`? #### Key features of `ActiveRecord::Encryption` 1. [Support for multiple encryption keys](https://gitlab.com/gitlab-org/gitlab/-/issues/490581) (rotating deterministic keys isn't supported yet, but support for it should be easy to add). - We might be able to work around the current limitation of deterministic encryption which prevents key rotation by specifying explicitly the key to use so that under the hood it'll use the `DerivedSecretKeyProvider` with `deterministic: true` option, i.e. `key: ActiveRecord::Encryption.config.deterministic_key, deterministic: true`. This is because of https://github.com/rails/rails/blob/v7.0.8.6/activerecord/lib/active_record/encryption/scheme.rb#L91 1. Ability to know what key was used to encrypt an attribute 1. Ability to list records encrypted with a specific key ### GitLab.com to Dedicated migration workflow [One of the goal of Cells is to allow customers to migrate from GitLab.com to Dedicated](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/cells/goals/#customer-can-migrate-from-gitlabcom-to-dedicated). In that scenario, it's preferable that the new Dedicated instance use different secrets than GitLab.com, for security purpose. This means the DB-stored encrypted data need to be re-encrypted with the Dedicated secrets. #### Pre-requisites 1. [Support for multiple encryption keys](https://gitlab.com/gitlab-org/gitlab/-/issues/490581) 1. Ability to know what key was used to encrypt an attribute 1. Ability to list records encrypted with a specific key These 3 pre-requisites would be fulfilled by using `ActiveRecord::Encryption` (except for rotating deterministic keys, but support for it should be easy to add), and should be implemented after https://gitlab.com/groups/gitlab-org/-/epics/15226+ is done. #### Workflow 1. Dedicated instance is set up with a new `db_key_base` key 1. GitLab.com `db_key_base` is added as a legacy key to `config/secrets.yml` (i.e. first key in the `db_key_base` array, as the last key is the current one, used to encrypt new data) - This will allow existing data to be decrypted with the GitLab.com key while the data is re-encrypted with the new key 2. A background process runs to re-encrypt any data that was encrypted with the legacy key, until the query condition returns no record - Pseudo code: ```ruby # ActiveRecord::Encryption doesn't support deterministic keys rotation at the moment, so the following would not work # legacy_deterministic_key_id = ActiveRecord::Encryption::DeterministicKeyProvider.new(ActiveRecord::Encryption.config.deterministic_key.first).encryption_key.id # ActiveRecord::Encryption do support non-deterministic keys rotation, so the following would work legacy_key_id = ActiveRecord::Encryption::DerivedSecretKeyProvider.new(ActiveRecord::Encryption.config.primary_key.first).encryption_key.id User.where("token->'h'->'i' ? :value", value: ::Base64.strict_encode64(legacy_key_id)).find_in_batches do |user| # When using ActiveRecord::Encryption user.encrypt # this forces the re-encryption of the `token` attribute # Use the following lines instead when using attr-encrypted # user.token = user.token # user.save! end ``` 3. The GitLab.com key can be removed from the Dedicated instance since all the data was re-encrypted with the new Dedicated key Notes: - We could imagine a new Admin page to track the background migration progress. The page would show for each legacy key ID how many records still need to be re-encrypted. ## Resources * Previous work: https://gitlab.com/gitlab-org/gitlab/-/issues/26243#note_1474177720 * [Migrate attr_encrypted to Rails 7 Active Record encrypts](https://pagertree.com/blog/migrate-attr_encrypted-to-rails-7-active-record-encrypts) * [Migrate from attr_encrypted to ActiveRecordEncryption](https://engineeringblog.wonolo.com/migrate-from-attrencrypted-to-activerecordencryption) * [Active Record Encryption - Rotating Keys](https://guides.rubyonrails.org/active_record_encryption.html#rotating-keys) * [How GitHub converts previously encrypted and unencrypted columns to ActiveRecord encrypted columns ](https://github.blog/2022-11-03-how-github-converts-previously-encrypted-and-unencrypted-columns-to-activerecord-encrypted-columns) * [How to Migrate attr_encrypted to Rails 7 Encryption ](https://gorails.com/episodes/migrate-attr_encrypted-to-rails-7-encryption)