Skip to content

Encrypt trigger tokens in DB

What does this MR do and why?

Trigger tokens are currently stored in clear text in our DB:

  • gitlabhq_production_ci DB
  • ci_triggers table
  • token field

Token value generation happens here without specific encryption or hashing taking place.

This MR adds 2 new columns into ci_triggers table. The batch background migration is populating the columns with encrypted values. A new attribute is also added to return the encrypted value: encrypted_token_tmp which will for now not be used.

Behaviour change: with every new trigger created, the encryption columns will be updated with encryption values for token. Token will still be the source of truth.

Issue https://gitlab.com/gitlab-org/gitlab/-/issues/372736

Recommended reviewing sequence:

  1. Updated to the Ci:Trigger model. Added attr_encrypted to encrypt token.
  2. View EncryptCiTriggerToken batch background migration which backfill the encryption values for all the triggers created pre this MR.
  3. View EncryptCiTriggerToken post migration that will orchestrate the batch migration.

Screenshots or screen recordings

AddCiTriggersEncryptedToken MIGRATION

**➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rails db:migrate                                        
main: == 20230126151622 AddCiTriggersEncryptedToken: migrating ======================
main: -- add_column(:ci_triggers, :encrypted_token, :binary)
main:    -> 0.0066s
main: -- add_column(:ci_triggers, :encrypted_token_iv, :binary)
main:    -> 0.0007s
main: == 20230126151622 AddCiTriggersEncryptedToken: migrated (0.0119s) =============

ci: == 20230126151622 AddCiTriggersEncryptedToken: migrating ======================
ci: -- add_column(:ci_triggers, :encrypted_token, :binary)
ci:    -> 0.0026s
ci: -- add_column(:ci_triggers, :encrypted_token_iv, :binary)
ci:    -> 0.0005s
ci: == 20230126151622 AddCiTriggersEncryptedToken: migrated (0.0099s) =============**
➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:ci VERSION=20230126151622 
ci: == 20230131123923 RaiseCiVariablesDefaultLimits: reverting ====================
ci: -- change_column_default(:plan_limits, :group_ci_variables, {:from=>30000, :to=>200})
ci:    -> 0.0834s
ci: -- change_column_default(:plan_limits, :project_ci_variables, {:from=>8000, :to=>200})
ci:    -> 0.0043s
ci: == 20230131123923 RaiseCiVariablesDefaultLimits: reverted (0.1023s) ===========

➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:main VERSION=20230126151622 
main: == 20230131123923 RaiseCiVariablesDefaultLimits: reverting ====================
main: -- change_column_default(:plan_limits, :group_ci_variables, {:from=>30000, :to=>200})
main:    -> 0.1415s
main: -- change_column_default(:plan_limits, :project_ci_variables, {:from=>8000, :to=>200})
main:    -> 0.0081s
main: == 20230131123923 RaiseCiVariablesDefaultLimits: reverted (0.1523s) ===========

EncryptCiTriggerToken POST MIGRATION

➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rails db:migrate:ci
ci: == 20230202131928 EncryptCiTriggerToken: migrating ============================
ci: == 20230202131928 EncryptCiTriggerToken: migrated (0.0485s) ===================
➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rails db:migrate:main
main: == 20230202131928 EncryptCiTriggerToken: migrating ============================
main: -- The migration is skipped since it modifies the schemas: [:gitlab_ci].
main: -- This database can only apply migrations in one of the following schemas: [:gitlab_main, :gitlab_shared, :gitlab_internal, :gitlab_pm, :gitlab_main_clusterwide].
main: == 20230202131928 EncryptCiTriggerToken: migrated (0.0015s) ===================

Screen_Shot_2023-02-02_at_16.28.35 Screen_Shot_2023-02-02_at_16.28.48

ROLLBACK

➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:main STEP=1
main: == 20230202131928 EncryptCiTriggerToken: reverting ============================
main: -- The migration is skipped since it modifies the schemas: [:gitlab_ci].
main: -- This database can only apply migrations in one of the following schemas: [:gitlab_main, :gitlab_shared, :gitlab_internal, :gitlab_pm, :gitlab_main_clusterwide].
main: == 20230202131928 EncryptCiTriggerToken: reverted (0.0014s) ===================
➜  gitlab git:(372736/Encrypt_trigger_tokens_in_DB_1) ✗ rake db:rollback:ci STEP=1  
ci: == 20230202131928 EncryptCiTriggerToken: reverting ============================
ci: == 20230202131928 EncryptCiTriggerToken: reverted (0.0319s) ===================

Screen_Shot_2023-02-02_at_16.33.17

How to set up and validate locally

  1. Create Ci::Trigger instances via rails console (eg 3)
project = Project.last
Ci::Trigger.new(project: project, owner:project.owner)
  1. Run migration (which will add 2 columns)
  2. Create some more Ci::Trigger instances via rails console (step 1)
    At this point, you will have some rows with and without the encrypted values.
  3. Open console and run

Gitlab::BackgroundMigration::EncryptCiTriggerToken.new(start_id:1, end_id:6, batch_table: 'ci_triggers', batch_column:'id', connection: Ci::ApplicationRecord.connection, sub_batch_size:1, pause_ms:250)
  1. Verify that the encrypted column can be read successfully.
Ci::Trigger.pluck(:id, :token, :encrypted_token)

To check the decrypted token

Ci::Trigger.last.encrypted_token_tmp
  1. In the psql console:
\c gitlabhq_development_ci
select token, encrypted_token, encrypted_token_iv from ci_triggers;

Screen_Shot_2023-02-03_at_14.04.06

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Kasia Misirli

Merge request reports

Loading