Project push rules: Migrate project push rules to dedicated table

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Label this issue

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Collaborate/take over this issue

Implementation plan

We want to use the push_rules table as a dedicated table for projects. Therefore, we should remove push_rules records belonging to a group or organization.

• Create a migration to destroy push_rules records where organization_id is populated.
• Create a migration to destroy push_rules records associated with a group (where push_rule_id is not null).

Expectations

• Follow this guideline for data migrations https://docs.gitlab.com/development/database_review/#preparation-when-adding-data-migrations
• Cover the migrations result with specs.
• Collaborate with the database team to get a review.

Next Steps

• Create a new table migration project_push_rules (w1)
• Add triggers to copy data from push_rules to project_push_rules similar to what we have for organization_push_rules and group_push_rules (w2)
• Write a BBM to copy historical data from push_rules to project_push_rules (w2)
  • Account for duplicates logic in BBM: !199307 (closed)
  • migrate missing records: !219705 (merged) (18.9)

Implementation Plan

project_push_rules table approach(this approach has been scrapped)

Step	Milestone	Weight	Done?
Create project_push_rules table	18.6		✅ Done
Sync project_push_rules on push_rules write operations	18.6		✅ Done
BBM to copy push_rules data to project_push_rules	18.7	2	✅ Done
Migrate duplicate push_rules to project_push_rules	18.7		✅ Done
BackfillMissingProjectPushRules migration	18.9	2	✅ Done
Reset project_push_rules id sequence to prevent UniqueViolation errors	18.10	2	MR ✅ Done
Write switch code (FF off)	18.10	3	Implementation plan
Enable read_project_push_rules FF on GitLab.com	18.11	1	✅

push_rules table approach(the new approach)

Tasks that must be completed for Cells(Gitlab.com first and then later for Self-managed)

	Task	milestone	effort	Done?
1	Revert project push rules read switch to use push_rules table	19.0	1	✅
2	Remove the trigger for group_push_rules	19.0	1	✅
2	Remove the trigger for organization_push_rules	19.1	1	✅
2	[Feature flag] Cleanup update_organization_push_rules	19.1	1	✅
2	Remove the trigger for project_push_rules	19.1	1	✅
3	Background migration to delete NULL project_id rows	19.1	2	✅
4	Finalise the BBM for only Gitlab.com	19.3(after the required stop(19.2))	1	✅
5	Add NOT NULL constraint on project_id in push_rules table	19.3	1	✅
6	Add application-level validation	19.3	1	✅
7	Validate the constraint for GitLab.com	19.3	1	✅
8	Add sharding key for push_rules	19.3	1	✅
9	Reschedule backfill for self-managed	19.3	1	✅

Tasks that can be deferred

	Task	milestone	effort	Done?
1	Clean up project_push_rules code/tests/table	19.4	2	✅
2	Finalize BBM for self-managed	19.6(after required stop 19.5)	1
3	Validate the constraint for self-managed	19.6(after required stop 19.5)	1