Move Sidekiq throttling functionality into the application

Placeholder issue:

@bbodenmiller asked:

Let's say project_update_repository_storage doesn't have a dedicated node, what's preventing moves from overloading server? How many will it try to run in parallel?

I replied:

@bbodenmiller at present, the jobs will be scheduled in the same way as any other sidekiq job would be, so potentially up the maximum concurrency of your sidekiq fleet.

At present there are no additional checks to ensure that the jobs are throttled. On GitLab.com we do this by isolating and limiting certain jobs (project_export, project_update_repository_storage etc).

There is a good case to be made for moving this functionality into the product, although the mechanism would need to be quite different from the one we use on GitLab.com. Possibly something like https://github.com/sensortower/sidekiq-throttled

Having a throttle mechanism on certain queues would make it easier for some of our customers to perform similar processes to what we do on GitLab.com using shard throttling. We should consider doing this.