Ability to start dedicated Sidekiq processes
This adds the ability to start Sidekiq processes for specific queues. See https://gitlab.com/gitlab-org/gitlab-ee/issues/1282 for more information.
Merge request reports
Activity
cc @pcarranza
Mentioned in issue #1282 (closed)
- Resolved by Yorick Peterse
- lib/gitlab/sidekiq_cluster.rb 0 → 100644
1 require 'open3' 2 3 module Gitlab 4 module SidekiqCluster 5 # The signals that should terminate both the master and workers. 6 TERMINATE_SIGNALS = %i(INT TERM) 7 8 # The signals that should simply be forwarded to the workers. 9 FORWARD_SIGNALS = %i(TTIN USR1 USR2 HUP) These were taken from https://github.com/mperham/sidekiq/wiki/Signals
- Resolved by Yorick Peterse
@marin From a packaging perspective, what are your requests when adding a new process to monitor by runit? For example: the script writes to a PID file if told to do so, but I'd like to know if there are more requests.
Edited by Yorick Peterse@yorickpeterse FYI, @marin is out for a few days. As far as I can tell, I think if we follow the conventions of the current runit processes, I think we'll be okay. Maybe @twk3 can chime in here too.
- Resolved by Yorick Peterse
Added 1 commit:
- 7aa95d04 - Ability to start dedicated Sidekiq processes
@stanhu Therein lies the problem: I'm not really aware of what those patterns/standards are other than processes not detaching themselves.
Added 1 commit:
- 9ceecb99 - Ability to start dedicated Sidekiq processes
added 710 commits
-
9ceecb99...05610a23 - 709 commits from branch
master
- c12ceebe - Ability to start dedicated Sidekiq processes
-
9ceecb99...05610a23 - 709 commits from branch
added 1 commit
- b137e5ba - Ability to start dedicated Sidekiq processes
@yorickpeterse I am having a bit of a problem following what you are trying to achieve (and ask). So I will write what I think is happening and we can get to the common ground from there:
The idea here is to be able to start up a process on demand and letting the supervisor handle the process from there on?
You are asking whether you need to do anything special for the supervisor? If that is the question, the answer is no. Supervisor watches the processes and daemonizes them automatically. If the process gets killed or stops, it will try to restart it and it will do that till the concept of time changes.
One question I have for you is, this is supposed to allow us to quickly spin up additional background workers and not as a replacement for the "main" sidekiq?
@marin The idea is to provide a tool that lets us start extra Sidekiq workers that only process a given set of queues. For GitLab.com we have a hack that uses tmux, but this of course doesn't work very pleasantly. These extra workers would be used to process important queues so processing timings are not affected by other queues.
To make this as easy as possible we came up with the idea of having some sort of wrapper script. This script would take a set of arguments (e.g. the queues) and spawn a number of
sidekiq
processes with the right settings. This is whatsidekiq-cluster
in this MR does. This can lead to a setup wheresidekiq
processes everything as usual, andsidekiq-cluster
starts two processes that processprocess_commit
andpost_receive
respectively (thus we'd have 3sidekiq
processes).One question I have for you is, this is supposed to allow us to quickly spin up additional background workers and not as a replacement for the "main" sidekiq?
It would not replace the main
sidekiq
process, instead it would complement it. This does mean that instead of something likesudo gitlab-ctl restart sidekiq
we'd also have to runsudo gitlab-ctl restart sidekiq-cluster
. For Omnibus this also means storing some configuration settings (e.g. the queues to use) somewhere, otherwiserestart
& friends won't know what settings to use. This could be something as simple as just a string passed tosidekiq-cluster
.Does that clear things up a bit?
Edited by Yorick Peterse@yorickpeterse Cool, thanks for explaining.