Encrypt trigger tokens in DB
What does this MR do and why?
Trigger tokens are currently stored in clear text in our DB:
-
gitlabhq_production_ciDB -
ci_triggerstable -
tokenfield
Token value generation happens here without specific encryption or hashing taking place.
This MR adds 2 new columns into ci_triggers table. The batch background migration is populating the columns with encrypted values. A new attribute is also added to return the encrypted value: encrypted_token_tmp which will for now not be used.
Behaviour change: with every new trigger created, the encryption columns will be updated with encryption values for token. Token will still be the source of truth.
Issue https://gitlab.com/gitlab-org/gitlab/-/issues/372736
Recommended reviewing sequence:
- Updated to the
Ci:Triggermodel. Added attr_encrypted to encrypt token. - View
EncryptCiTriggerTokenbatch background migration which backfill the encryption values for all the triggers created pre this MR. - View
EncryptCiTriggerTokenpost migration that will orchestrate the batch migration.
Screenshots or screen recordings
AddCiTriggersEncryptedToken MIGRATION
**➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rails db:migrate
main: == 20230126151622 AddCiTriggersEncryptedToken: migrating ======================
main: -- add_column(:ci_triggers, :encrypted_token, :binary)
main: -> 0.0066s
main: -- add_column(:ci_triggers, :encrypted_token_iv, :binary)
main: -> 0.0007s
main: == 20230126151622 AddCiTriggersEncryptedToken: migrated (0.0119s) =============
ci: == 20230126151622 AddCiTriggersEncryptedToken: migrating ======================
ci: -- add_column(:ci_triggers, :encrypted_token, :binary)
ci: -> 0.0026s
ci: -- add_column(:ci_triggers, :encrypted_token_iv, :binary)
ci: -> 0.0005s
ci: == 20230126151622 AddCiTriggersEncryptedToken: migrated (0.0099s) =============**
➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:ci VERSION=20230126151622
ci: == 20230131123923 RaiseCiVariablesDefaultLimits: reverting ====================
ci: -- change_column_default(:plan_limits, :group_ci_variables, {:from=>30000, :to=>200})
ci: -> 0.0834s
ci: -- change_column_default(:plan_limits, :project_ci_variables, {:from=>8000, :to=>200})
ci: -> 0.0043s
ci: == 20230131123923 RaiseCiVariablesDefaultLimits: reverted (0.1023s) ===========
➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:main VERSION=20230126151622
main: == 20230131123923 RaiseCiVariablesDefaultLimits: reverting ====================
main: -- change_column_default(:plan_limits, :group_ci_variables, {:from=>30000, :to=>200})
main: -> 0.1415s
main: -- change_column_default(:plan_limits, :project_ci_variables, {:from=>8000, :to=>200})
main: -> 0.0081s
main: == 20230131123923 RaiseCiVariablesDefaultLimits: reverted (0.1523s) ===========
EncryptCiTriggerToken POST MIGRATION
➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rails db:migrate:ci
ci: == 20230202131928 EncryptCiTriggerToken: migrating ============================
ci: == 20230202131928 EncryptCiTriggerToken: migrated (0.0485s) ===================
➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rails db:migrate:main
main: == 20230202131928 EncryptCiTriggerToken: migrating ============================
main: -- The migration is skipped since it modifies the schemas: [:gitlab_ci].
main: -- This database can only apply migrations in one of the following schemas: [:gitlab_main, :gitlab_shared, :gitlab_internal, :gitlab_pm, :gitlab_main_clusterwide].
main: == 20230202131928 EncryptCiTriggerToken: migrated (0.0015s) ===================
ROLLBACK
➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:main STEP=1
main: == 20230202131928 EncryptCiTriggerToken: reverting ============================
main: -- The migration is skipped since it modifies the schemas: [:gitlab_ci].
main: -- This database can only apply migrations in one of the following schemas: [:gitlab_main, :gitlab_shared, :gitlab_internal, :gitlab_pm, :gitlab_main_clusterwide].
main: == 20230202131928 EncryptCiTriggerToken: reverted (0.0014s) ===================
➜ gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:ci STEP=1
ci: == 20230202131928 EncryptCiTriggerToken: reverting ============================
ci: == 20230202131928 EncryptCiTriggerToken: reverted (0.0319s) ===================
How to set up and validate locally
- Create
Ci::Triggerinstances via rails console (eg 3)
project = Project.last
Ci::Trigger.new(project: project, owner:project.owner)
- Run migration (which will add 2 columns)
- Create some more
Ci::Triggerinstances via rails console (step 1)
At this point, you will have some rows with and without the encrypted values. - Open console and run
Gitlab::BackgroundMigration::EncryptCiTriggerToken.new(start_id:1, end_id:6, batch_table: 'ci_triggers', batch_column:'id', connection: Ci::ApplicationRecord.connection, sub_batch_size:1, pause_ms:250)
- Verify that the encrypted column can be read successfully.
Ci::Trigger.pluck(:id, :token, :encrypted_token)
To check the decrypted token
Ci::Trigger.last.encrypted_token_tmp
- In the psql console:
\c gitlabhq_development_ci
select token, encrypted_token, encrypted_token_iv from ci_triggers;
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.



