Move CI pipelines migration to a lower stage during Gitlab Migration
What does this MR do and why?
Mentions #358776 (closed)
This MR:
- Fixes exception during Project Migration when using Gitlab Migration
PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_merge_requests_on_target_project_id_and_iid" DETAIL: Key (target_project_id, iid)=(697, 6) already exists.
- Moves
BulkImports::Projects::Pipelines::CiPipelinesPipeline
from stage 4 to stage 5 in order to remove this potential race condition
This was happening due to merge requests & ci_pipelines migrations were in one stage (4), which could lead to 2 sidekiq jobs running at the same time. CI Pipelines reference MRs and create them during the import if they do not exist. IF CI Pipelines migration created one MR and then MRs migration tries to create the same MR - the exception occurs.
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
-
Feature.enable(:bulk_import)
. -
Feature.enable(:bulk_import_projects)
. - Create a top level group and project in it.
- Create an MR and CI pipeline that references that MR
- Go to
/groups/new#import-group-pane
page and enter instance url and access token (needs to beapi
&read_repository
scope). - Select newly created group and click Import.
- Wait for Group import to complete and verify imported project.
- Verify that MR and CI pipelines are migrated
- Check
bulk_import_failures
table and verify noPG::UniqueViolation
exception is raised & captured
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by George Koltsov