Add background worker to process scheduled member deletions
Summary
In order to handle the deletion of many member records as efficiently as possible and without timeout, we are introducing a scheduled deletion process that will schedule member deletions via a new table and asynchronously process those scheduled deletions in a controlled and scalable manner.
See #467281 (comment 2135360209) for more details and previous discussion.
Proposal
To achieve this, the effort has been broken down into multiple steps:
# | Step | Issue/MR | Notes |
---|---|---|---|
1 | Add a new model for tracking member deletion | #501247 (closed) | |
2 | Add a background worker to process the scheduled deletions | this one! |
|
3 | Update relevant APIs | #501249 (closed) | |
4 | Update the UI/UX | #480302 (closed) | |
5 | Review and iterate | #501250 (closed) |
Implementation detail
Create a background worker that processes records in the newly added table and deletes user memberships to a group hierarchy accordingly:
- One job (limited capacity worker?) running at a time
- Log relevant details (noted in this comment)
- Delete (or mark as complete?) the DB record
- We do something similar for AggregationSchedules via the
PruneAggregationsScheduleWorker
Edited by Vijay Hawoldar