Split ghost user migration queue into human and non-human
What does this MR do and why?
In #583853 it was reported that human user account deletions have been delayed because number of deletions for bot user accounts has increased. The delay for human user account deletions is happening because GitLab asynchronously process user account deletions by the single queue/Users::MigrateRecordsToGhostUserInBatchesWorker worker.
This MR splits Users::MigrateRecordsToGhostUserInBatchesWorker worker into two Users::MigrateHumanRecordsToGhostUserInBatchesWorker and Users::MigrateNonHumanRecordsToGhostUserInBatchesWorker workers to process human user accounts separately to prevent this issue and to ensure that bot user deletions spikes don't affect human user deletions in the future. The worker split is controlled by a split_ghost_user_migration_queue_into_human_and_non_human FF , which is disabled by default.
Database
To be able efficiently identify first 1000 human and non-human users that need to be deleted as per consume_after, id order in the new workers, this MR
- adds
user_typecolumn toghost_user_migrationstable. - sets
user_typefor newghost_user_migrationsin the application code. - backfills
user_typevalue for existingghost_user_migrationsrecords. - adds indexes needed to efficiently identify first 1000 human and non-human users that need to be deleted as per
consume_after, idorder.- Query plans: !235048 (comment 3381590084), !235048 (comment 3381597441).
References
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.