Skip to content

Backfill project_keys from project_key in jira_tracker_data table

Ivane Gkomarteli requested to merge ig-backfill-project_keys into master

What does this MR do and why?

This merge request introduces a new batched background migration to populate the project_keys column in the jira_tracker_data table with the project_key value for records that don't have project_keys yet

The new project_keys column will be used to replace project_key, and fetch issues from jira.

issue: #440430 (closed)

Query reports: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27488/commands/85624 https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27488/commands/85635

Records cound to be updated: 19977

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

Database migration

rails db:migrate
main: == [advisory_lock_connection] object_id: 122140, pg_backend_pid: 18244
main: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: migrating ==========
main: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: migrated (0.0415s) =

main: == [advisory_lock_connection] object_id: 122140, pg_backend_pid: 18244
ci: == [advisory_lock_connection] object_id: 122580, pg_backend_pid: 18246
ci: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: migrating ==========
ci: -- The migration is skipped since it modifies the schemas: [:gitlab_main].
ci: -- This database can only apply migrations in one of the following schemas: [:gitlab_ci, :gitlab_internal, :gitlab_shared].
ci: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: migrated (0.0073s) =

ci: == [advisory_lock_connection] object_id: 122580, pg_backend_pid: 18246
rails db:rollback:ci db:rollback:main
ci: == [advisory_lock_connection] object_id: 121840, pg_backend_pid: 19003
ci: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: reverting ==========
ci: -- The migration is skipped since it modifies the schemas: [:gitlab_main].
ci: -- This database can only apply migrations in one of the following schemas: [:gitlab_ci, :gitlab_internal, :gitlab_shared].
ci: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: reverted (0.0068s) =

ci: == [advisory_lock_connection] object_id: 121840, pg_backend_pid: 19003
main: == [advisory_lock_connection] object_id: 122620, pg_backend_pid: 19247
main: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: reverting ==========
main: == 20240402143848 QueueBackfillJiraTrackerDataProjectKeys: reverted (0.0327s) =

main: == [advisory_lock_connection] object_id: 122620, pg_backend_pid: 19247

How to set up and validate locally

  1. Integrations::JiraTrackerData.last.update(project_keys: 'TEST', project_keys: []) - Set project_key for a jira_tracker_data record and set project_keys to []
  2. Run the migration in this MR
  3. Integrations::JiraTrackerData.last.project_keys should be ['TEST']
Edited by Ivane Gkomarteli

Merge request reports