Make repository pull mirroring not depend on sidekiq queue sizes
Follow up from: gitlab-com/gl-infra/scalability#995 (closed)
At the time of writing, the UpdateAllMirrorsWorker
uses the queue size of the ProjectImportScheduleWorker
before possibly rescheduling itself.
I believe this is done to spread the load out a bit more, making sure we don't schedule more ProjectImportScheduleWorker
s and the following RepositoryUpdateMirrorWorker
s than we can chew.
In gitlab-com/gl-infra&447 (closed) we're moving from having a queue per worker, to jobs for multiple workers being scheduled on a single queue.
This means that checking the queue size would no longer be valid for checking if the UpdateAllMirrorsWorker
can continue scheduling jobs.
In the short term, we're keeping the ProjectImportScheduleWorker
on its own queue so this mechanism stays correct. But ideally we'd figure out a better way to do this.