Sidekiq CNG container should launch Sidekiq through bin/sidekiq-cluster
While working on &6409 (closed), we discovered that the proposed change of moving Sidekiq metrics exporting from the worker processes to a separate process would not currently work with our CNG Sidekiq container, because it merely uses the sidekiq-cluster
scripts to obtain arguments to then launch a single Sidekiq worker.
We need to change this to instead run sidekiq-cluster
with a single worker, so that the metrics server that will run in a separate process under sidekiq-cluster
is launched and supervised.
Moreover and as summarized in &6409 (comment 725349833), we think we should avoid having these environment specific configuration splits anyway, since it makes it hard to reason about what will run in which kind of deployment, and mistakes are easy to make.
Running sidekiq-cluster
on top of the single worker process will come with some minor memory overhead of around 30MB, due to the parent process being a separate Ruby process with which workers share no or little memory. We think this is acceptable and is likely to remain steady over time since outside of running a control loop, the sidekiq-cluster
process does not perform any work.