You need to sign in or sign up before continuing.
Backfill data from merge requests table into compliance violations table
What does this MR do and why?
-
NOTE: This MR is currently opened against !107998 (merged) and I'll switch it to
master
once that MR is merged. - Creates a background migration to copy the following data
- Copy
title
,target_branch
andtarget_project_id
frommerge_requests
table intomerge_requests_compliance_violations
table. - Copy
merged_at
column frommerge_request_metrics
table intomerge_requests_compliance_violations
table.
- Copy
- This is an irreversible migration, however, in case we find an issue with the migrated data after deploying on production, we can write another migration and override these columns again. The new columns defined
merge_requests_compliance_violations
table in !107998 (merged) are not used in the application code as of now. - Affecting 3215346 rows.
gitlabhq_dblab=# select count(*) from merge_requests_compliance_violations;
count
---------
3215346
(1 row)
Database queries
UPDATE
"merge_requests_compliance_violations"
SET
"merged_at" = '2022-10-27 06:11:43.694453',
"target_project_id" = 17,
"title" = 'Ab sunt fuga ut voluptatum atque voluptatem.',
"target_branch" = 'main'
WHERE
"merge_requests_compliance_violations"."id" = 648
SELECT
"merge_requests_compliance_violations".*
FROM
"merge_requests_compliance_violations"
WHERE
"merge_requests_compliance_violations"."id" >= 1
AND "merge_requests_compliance_violations"."id" < 500
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
- Before running the migration ensure that you have a few rows in
merge_requests_compliance_violations_table
by running the following in rails console
MergeRequests::ComplianceViolation.count
- In case you don't have any rows you can seed the data by running the following in your shell
FILTER=compliance_report_violations rake db:seed_fu
- Verify that
title
,target_project_id
,target_branch
andmerged_at
columns in themerge_requests_compliance_violations
table isnil
. If not you can forcefully make them nil by running the following in rails console
MergeRequests::ComplianceViolation.update_all(:title => nil)
- Check the count of rows with
nil
title by running
MergeRequests::ComplianceViolation.where(:title => nil).count
- Queue background migration by running
bundle exec rails db:migrate
- Verify that the batched background migration
BackfillComplianceViolations
is created by running the following in rails console.
Gitlab::Database::BackgroundMigration::BatchedMigration.find_by_job_class_name('BackfillComplianceViolations')
- Visit the background_migrations admin dashboard and verify that the background migration is completed successfully.
- Ensure that we have no rows with
nil
values for any of the above columns. You can again check the count by running the following in rails console
MergeRequests::ComplianceViolation.where(:title => nil).count # This will check for nil values for the `title` column
- Ensure manually that the
title
is populated with the correct data i.e. the MR's title for which the compliance violation record is created. Repeat this step for other columns as well.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #385500 (closed)
Edited by Huzaifa Iftikhar