Split ghost user migration queue into human and non-human

What does this MR do and why?

In #583853 it was reported that human user account deletions have been delayed because number of deletions for bot user accounts has increased. The delay for human user account deletions is happening because GitLab asynchronously process user account deletions by the single queue/Users::MigrateRecordsToGhostUserInBatchesWorker worker.

This MR splits Users::MigrateRecordsToGhostUserInBatchesWorker worker into two Users::MigrateHumanRecordsToGhostUserInBatchesWorker and Users::MigrateNonHumanRecordsToGhostUserInBatchesWorker workers to process human user accounts separately to prevent this issue and to ensure that bot user deletions spikes don't affect human user deletions in the future. The worker split is controlled by a split_ghost_user_migration_queue_into_human_and_non_human FF , which is disabled by default.

Database

To be able efficiently identify first 1000 human and non-human users that need to be deleted as per consume_after, id order in the new workers, this MR

  • adds user_type column to ghost_user_migrations table.
  • sets user_type for new ghost_user_migrations in the application code.
  • backfills user_type value for existing ghost_user_migrations records.
  • adds indexes needed to efficiently identify first 1000 human and non-human users that need to be deleted as per consume_after, id order.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Bogdan Denkovych

Merge request reports

Loading