Skip to content

[Data migration] Sync Approver/ApproverGroup with ApprovalRule/ApprovalRuleMember

Mark Chao requested to merge 1979-1-2-multi-block-approval-data-migration into master

CE port at https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24032

What does this MR do?

See https://gitlab.com/gitlab-org/gitlab-ee/issues/8647 for overall description.

SVG flow chart:
migration

A main background migration, MigrateApproverToApprovalRules, will

  1. Create counterpart of Approver ApproverGroup in approval rules.
  2. If MR is merged
  3. copy project level rule as MR level rule.
  4. Map Approval with rule

A few hooks are also added so Approver/ApproverGroup changes are synced with new models.

Estimation on migration time

Projects

There are estimated to be around 50,000 projects. This would take 15 minutes in total. This is run sync.

Merge Requests

  • Run async
  • There are around 18,000,000 MRs.
  • 1000 MR can take 15 seconds to process.
  • job to job interval is 2 minutes

A google spreadsheet is used to calculate batch size https://docs.google.com/spreadsheets/d/1Kowl8ngPQhxjZHuaSorSG7b6hOEAClmqFPw2rt2RNc0/edit?usp=sharing

The batch size of 3000 is chosen, in order to finish migration under 12 days.

How is data being persisted before/after merge

The way models are finalized after merge can vary a bit. To illustrate the 3 scenarios, let's assume when MR is still open, the yellow rows are present:

1. When MR rule override exists

1979-model-mr-level.svg

The blue rows are created once the MR is merged, recording which user approved which rule.

2. Project rule without MR rule override

1979-model-project-level

The blue rows are created once the MR is merged, which are copying the project level rules as MR rules, and then approved_approvers.

3. Project rule without MR rule override, with code owner

Code owner rule always exists at MR level:

1979-model-project-level-with-co

What are the relevant issue numbers?

#8647 (closed)

Does this MR meet the acceptance criteria?

Closes #1979 (closed)

Edited by Mark Chao

Merge request reports