Allow config of sidekiq-cluster with query
What does this MR do?
This adds a new command-line option to sidekiq-cluster
- --experimental-queue-selector
. When this flag is set, it enables queue groups to be selected as described in gitlab-com/gl-infra/scalability#45 (closed), and the documentation here. For a very simple example, the below starts two processes. One is handling all memory-bound issue tracking queues (just export_csv
), and the other is handling all memory-bound source code management queues (just project_export
):
$ ee/bin/sidekiq-cluster --dryrun --experimental-queue-selector 'resource_boundary=memory&feature_category=issue_tracking' 'resource_boundary=memory&feature_category=source_code_management'
2020-02-06T15:19:41.862Z 38268 TID-oxggy2srk INFO: Starting cluster with 2 processes
Sidekiq command: ["bundle", "exec", "sidekiq", "-c 2", "-edevelopment", "-gqueues: export_csv", "-r/Users/seanmcgivern/Code/gdk/gitlab", "-qexport_csv,1"]
Sidekiq command: ["bundle", "exec", "sidekiq", "-c 2", "-edevelopment", "-gqueues: project_export", "-r/Users/seanmcgivern/Code/gdk/gitlab", "-qproject_export,1"]
Omnibus MR: omnibus-gitlab!3920 (merged)
Closes gitlab-com/gl-infra/scalability#45 (closed)
Does this MR meet the acceptance criteria?
Conformity
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
Edited by 🤖 GitLab Bot 🤖