Invalid members migration follow-up
Proposal
The plan is:
- Add a temp index on null member namespace id so we can index over it later: !99212 (merged)
✅ - Delete the migration that used the less performant column: !99231 (merged)
- Create new migration using the more performant column and index with a later timestamp !99467 (merged)
- Finalize migration !99080 (merged)
- Drop the temp index
Some other notes that aren't necessarily the plan but are helpful:
gitlab-com/gl-infra/production#7779 (closed)
- Cancel
db/post_migrate/20220809002011_schedule_destroy_invalid_group_members.rb
anddb/post_migrate/20220901035725_schedule_destroy_invalid_project_members.rb
migrations as these do not seem to be succeeding.
A no-op migration would not help in this case as the migration was already scheduled. I think we need to prepare a migration that either removes the scheduled migrations from DB i.e. delete_batched_background_migration
or my personal opinion add a new aborted
status for the migrations so that we keep the migration for historical tracking. However adding a new status is a bit more work and perhaps needs to be coordinated with DB team.
- Add back the removed partial index on
member_namespace_id
with the condition ofmember_namespace_id is null
Undo !98387 (merged) changes
- Write a new migration that cleans records
where member_namespace_id is null
which should be much more performant as we should be having very few invalid records.
Reimplement !95881 (merged) and !80253 (merged) using member_namespace_id
instead of id
- Make sure that is successful and add back the finalize cleanup migration.
Reimplement !97781 (merged) using member_namespace_id
instead of id
gitlab-com/gl-infra/production#7779 (comment 1115925200)
- We probably need to restore the old index (gitlab-com/gl-infra/production#7779 (comment 1109024224))
- Create a new MR containing the removed migration code (use new migration timestamps).
- Run the DB-testing pipeline.
- Verify that the query performance is bad. (ping me on the MR)
- Fix the DB queries (possible fix is mentioned here: gitlab-com/gl-infra/production#7779 (comment 1109024224))
- Merge
Next milestone:
- Finalize the DB migration.
- Drop the index.