Separate gitlab-cluster execution from the standard sidekiq fleet
A continuation of https://gitlab.com/gitlab-com/infrastructure/issues/1810#note_31371871
What we can do to take this under control is split the fleet.
We can start sidekiq-cluster with a specific configuration in any host we want, what we don't have is a way of setting this is up in chef because our roles are still too monolithical.
What we need to do is:
- Build a separate fleet for sidekiq cluster and set the queues we want to run on each host independent one from each other, even reaching at the attribute level in chef.
- Disable sidekiq-cluster in the standard sidekiq fleet.
- Rinse and repeat until queues work as needed and load is under control.
Edited by John Northrup