Cleanup for converted ci_stages.pipeline_id conversion
What does this MR do and why?
Cleanup for converted ci_stages.pipeline_id conversion
This is to step to remove the trigger and old integer column See https://docs.gitlab.com/ee/development/database/avoiding_downtime_in_migrations.html#remove-the-trigger-and-old-integer-columns-release-n--2
- Backfill is done at !126650 (merged) and !127471 (merged)
- Index is created at !127471 (merged) and !130917 (merged)
- Foreign key is created at !130901 (merged) and !132956 (merged)
- Columns are swapped at !133307 (merged)
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #422816 (closed)
Merge request reports
Activity
changed milestone to %16.7
assigned to @tianwenchen
- A deleted user
added databasereview pending label
- Resolved by Michał Zając
1 Warning There were no new or modified feature flag YAML files detected in this MR. If the changes here are already controlled under an existing feature flag, please add
the feature flagexists. Otherwise, if you think the changes here don't need
to be under a feature flag, please add the label feature flagskipped, and
add a short comment about why we skipped the feature flag.For guidance on when to use a feature flag, please see the documentation.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend @tyleramos
(UTC-5, 16 hours behind author)@mksionek
(UTC+1, 10 hours behind author)database @dskim_gitlab
(UTC+11, same timezone as author)@l.rosa
(UTC-3, 14 hours behind author)Please check reviewer's status!
Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger - A deleted user
added Data WarehouseImpact Check label
added 167 commits
-
d88435c2...8d089983 - 166 commits from branch
master
- 85b35f09 - Cleanup for converted ci_stages.pipeline_id conversion
-
d88435c2...8d089983 - 166 commits from branch
- A deleted user
added backend label
Allure report
allure-report-publisher
generated test report!e2e-test-on-gdk:
test report for c7c87abd expand test summary
+-----------------------------------------------------------------------+ | suites summary | +------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------+--------+--------+---------+-------+-------+--------+ | Verify | 31 | 0 | 0 | 0 | 31 | ✅ | | Data Stores | 23 | 0 | 0 | 0 | 23 | ✅ | | Monitor | 8 | 0 | 0 | 0 | 8 | ✅ | | Create | 52 | 0 | 9 | 0 | 61 | ✅ | | Manage | 0 | 0 | 1 | 0 | 1 | ➖ | | Govern | 57 | 0 | 0 | 0 | 57 | ✅ | | Plan | 55 | 0 | 0 | 0 | 55 | ✅ | | Framework sanity | 0 | 0 | 1 | 0 | 1 | ➖ | | Package | 0 | 0 | 1 | 0 | 1 | ➖ | +------------------+--------+--------+---------+-------+-------+--------+ | Total | 226 | 0 | 12 | 0 | 238 | ✅ | +------------------+--------+--------+---------+-------+-------+--------+
e2e-package-and-test:
test report for c7c87abd expand test summary
+-------------------------------------------------------------+ | suites summary | +--------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +--------+--------+--------+---------+-------+-------+--------+ | Verify | 138 | 0 | 27 | 0 | 165 | ✅ | | Create | 131 | 0 | 12 | 0 | 143 | ✅ | +--------+--------+--------+---------+-------+-------+--------+ | Total | 269 | 0 | 39 | 0 | 308 | ✅ | +--------+--------+--------+---------+-------+-------+--------+
Database migrations (on the main database)
1 Warnings 20231120070345 - CleanupCiStagesPipelineIdBigint had a query that exceeded timing
guidelines. Run time should not exceed 100ms, but it was 100.84ms. Please consider possible options
to improve the query performance.ALTER TABLE "ci_stages" DROP COLUMN
"pipeline_id_convert_to_bigint"Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20231120070345 - CleanupCiStagesPipelineIdBigint Post deploy 2.2 s -32.00 KiB Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 3 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20231120070345 - CleanupCiStagesPipelineIdBigint - Type: Post deploy
- Duration: 2.2 s
- Database size change: -32.00 KiB
Calls Total Time Max Time Mean Time Rows Query 1 100.8 ms 100.8 ms 100.8 ms 0 ALTER TABLE "ci_stages" DROP COLUMN "pipeline_id_convert_to_bigint"
1 16.5 ms 16.5 ms 16.5 ms 0 LOCK TABLE ci_pipelines, ci_stages IN ACCESS EXCLUSIVE MODE
1 9.4 ms 9.4 ms 9.4 ms 0 DROP TRIGGER IF EXISTS trigger_07bc3c48f407 ON "ci_stages"
1 3.8 ms 3.8 ms 3.8 ms 0 DROP FUNCTION IF EXISTS trigger_07bc3c48f407()
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for CleanupCiStagesPipelineIdBigint
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 3 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20231109124031 - EnsureIdUniquenessForPCiBuildsV5 Post deploy 3.1 s +0.00 B 20231114053330 - CleanupCiPipelinesAutoCanceledByIdBigint Post deploy 2.9 s -8.00 KiB 20231115104943 - RemoveServiceAccessTokensCategoryColumn Post deploy 1.7 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-2548311-11970317-main
2023-11-21T03:47:22Z 2023-11-21T01:18:55Z 2023-11-21 15:53:08 +0000 database-testing-2548311-11970317-ci
2023-11-21T03:47:22Z 2023-11-21T00:47:05Z 2023-11-21 15:53:08 +0000 Database migrations (on the ci database)
1 Warnings 20231120070345 - CleanupCiStagesPipelineIdBigint had a query that exceeded timing
guidelines. Run time should not exceed 100ms, but it was 100.26ms. Please consider possible options
to improve the query performance.ALTER TABLE "ci_stages" DROP COLUMN
"pipeline_id_convert_to_bigint"Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20231120070345 - CleanupCiStagesPipelineIdBigint Post deploy 52.8 s -160.15 GiB Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 5 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20231120070345 - CleanupCiStagesPipelineIdBigint - Type: Post deploy
- Duration: 52.8 s
- Database size change: -160.15 GiB
Calls Total Time Max Time Mean Time Rows Query 1 100.3 ms 100.3 ms 100.3 ms 0 ALTER TABLE "ci_stages" DROP COLUMN "pipeline_id_convert_to_bigint"
1 16.0 ms 16.0 ms 16.0 ms 0 DROP TRIGGER IF EXISTS trigger_07bc3c48f407 ON "ci_stages"
1 3.3 ms 3.3 ms 3.3 ms 0 LOCK TABLE ci_pipelines, ci_stages IN ACCESS EXCLUSIVE MODE
1 0.2 ms 0.2 ms 0.2 ms 0 DROP FUNCTION IF EXISTS trigger_07bc3c48f407()
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for CleanupCiStagesPipelineIdBigint
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 5 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20231109124031 - EnsureIdUniquenessForPCiBuildsV5 Post deploy 3.6 s +0.00 B 20231114053330 - CleanupCiPipelinesAutoCanceledByIdBigint Post deploy 3.3 s -6.69 GiB 20231115104943 - RemoveServiceAccessTokensCategoryColumn Post deploy 2.6 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-2548311-11970317-main
2023-11-21T03:47:22Z 2023-11-21T01:18:55Z 2023-11-21 15:53:08 +0000 database-testing-2548311-11970317-ci
2023-11-21T03:47:22Z 2023-11-21T00:47:05Z 2023-11-21 15:53:08 +0000
Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic
- A deleted user
added database-testing-automation label
- Resolved by Marius Bobin
- Resolved by Marius Bobin
@eurie could you help to do the first review? If it looks good to you, could you pass it onto
@mbobin
? Thanks in advance!
requested review from @eurie
- Resolved by Marius Bobin
- Resolved by Marius Bobin
LGTM @tianwenchen!
I left one question for the maintainer to weigh in on. Also, can you add the output from the migrate up & down to the description for the maintainer?
@Quintasan can you do the database maintainer review?
added databasereviewed label and removed databasereview pending label