Switch Reference Architectures to use sidekiq-cluster and test
With the work at gitlab-com/gl-infra&181 (closed) we're looking forward to use sidekiq-cluster
by default for all environments (including self-hosted).
As spoken with @grantyoung, the main limitation for setting up the environments to use sidekiq-cluster
were having to pass all queues to the sidekiq-cluster
script, which we've now changed to support the bin/sidekiq-cluster *
(also being able to pass the queue_groups
config https://docs.gitlab.com/ee/administration/operations/extra_sidekiq_processes.html#starting-extra-processes).
Seems like we'll need to change / create a new role for sidekiq when building the environment. I believe having a sidekiq (cluster) process for each node would be fine.
No change in performance is expected given sidekiq-cluster
is basically a wrapper for sidekiq
, and we'll be starting a single sidekiq
process per node still.
cc @grantyoung