Sidekiq-cluster emit startup command option
This idea sprang from a discussion with @jarv.
One of our goals, whenever possible, in our move to Kubernetes is to avoid replicating in the application what Kubernetes provides out the box.
This includes process supervision.
In some ways sidekiq-cluster
is basic, limited-feature process supervisor. In kubernetes, we should avoid this. This is doubly pertinent since, in k8s, sidekiq-cluster
will only supervise a single sidekiq process. Scaling horizontally with additional sidekiq processes should be done with additional pods, rather that additional sidekiq processes in a single pod.
Unfortunately, we still need other features in sidekiq-cluster, primarily the ability to resolve queue selector syntax into a set of queues.
Proposal
Include a --emit-command
option on sidekiq-cluster
.
When this boolean flag is included, sidekiq-cluster
will emit the appropriate command to start Sidekiq to stdout, before successfully exiting.
For example:
$ sidekiq-cluster --emit-command --concurrency 5 --experimental-queue-selector "resource_boundary!=cpu&latency_sensitive=true"
bundle exec sidekiq --concurrent 5 --queues ... # etc
$
Using this approach, sidekiq could be booted, without a sidekiq-cluster supervisor process with the following (pseudo-shell-script)
#!/bin/sh
command=$(sidekiq-cluster --emit-command --concurrency 5 --experimental-queue-selector "resource_boundary!=cpu&latency_sensitive=true")
exec $(command)