Scale default number of processes dedicated to Sidekiq
Summary
A default Omnibus installation will deploy a single Sidekiq catch-all queue.
This is easy to overwhelm with jobs and when this happens customers will reach out for support as they experience high load or strange behavior such as pipelines and merge requests not updating, geo not replicating etc.
Proposal
We could increase the number of Sidekiq processes per number of CPU available. As an example, for each 10 CPU's the customer has, we could add another one dedicated to Sidekiq with sidekiq['queue_groups'] = ['*'] * (<number of cpus / 10>)
.
Note: We could also introduce further optimization such as dedicating these processes to specific queues depending on configuration such as if the customer is using Geo.