Parallelize LooseForeignKeys::CleanupWorker
This is 1 way we may be able to improve the performance of cleaning up loose foreign keys if we find ourselves reaching the the MAX_RUNTIME=30s
too often. This is an alternative approach to the one described in #343550 (closed) .
The idea is to shard the LooseForeignKeys::CleanupWorker
by parent_table
so that every minute we run many of these in parallel that are processing different sets of parent_table
lists since these are all independent. We could shard by taking a checksum of the table name and split them into N groups for N workers. Since we have ~60 parent tables this should mean we can easily split into N=10 workers and get a substantial reduction in the total time of all workers.
We may only need to invest time in this if we see scaling problems in: