Consider adding a delay between executing background jobs after picking them up
Right now we typically schedule background migrations with an interval of a few minutes. This ensures that if they all run OK they don't all run at once.
Unfortunately if jobs have to be retried or somehow aren't processed fast enough the interval/timeout could expire. This can lead to many jobs getting picked up in sequence around the same time. This in turn can lead to high database load.
We should investigate if it's possible for workers to not poll the background_migration
queue for a while (e.g. 5 minutes) if they finished a job. This ensures that even if 10 000 jobs are to be processed at once, only a few of them run every 5 minutes.