Extract Sidekiq query selector matching logic to a new class
What does this MR do?
GitLab supports Sidekiq queue selector to start a Sidekiq cluster listening for a set of queues matching some input queries. If filters the list of workers accordingly based on the worker's metadata such as resource boundary, feature category, urgency, etc. Here are some queries we are using on GitLab.com:
resource_boundary=memoryfeature_category=database&urgency=throttledfeature_category=gitaly&urgency=throttled- And so on
Internally, each worker belongs to a dedicated queue generated from the worker name. The above queries are translated into a list of corresponding queues so that the Sidekiq processes can listen to. This approach soon exposes some performance disadvantages. One of the most significant one is that Sidekiq processes have to watch for many queues. At moment, the busiest shard is watching at around 438 queues.
We are moving forward to reduce the number of queues by routing a job into a pre-defined queue at schedule time. The routing mechanism is the same as the queue selector. We maintain an array of tuples to pick the queue based on the query:
[
["resource_boundary=memory", "memory-bound"],
["feature_category=database&urgency=throttled", "database-throttled"],
["feature_category=gitaly&urgency=throttled", "gitaly-throttled"]
]
However, the whole queue selecting logic is exclusive and tight coupling to Sidekiq CLI. This MR is extract them out, clean up, and prepare for the core routing implementation in the next MR. After this MR is deployed, we expect nothing gonna change, operation-wise.
Please read gitlab-com/gl-infra/scalability#1016 and gitlab-com/gl-infra&447 (closed) for more information.
Screenshots (strongly suggested)
- Apart from the unit tests, I do a series of manual tests. For each GitLab.com's production shards, I compare the list of workers matched by that shard's selector on local environment, and active workers on production. At the first glance, all of them matched.
-
*selector
- Without using queue-selector
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because this is an internal change, nothing changes from the user side.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec -
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team








