Attribute all Sidekiq workers according to their workloads
Merge sequence: !18462 (merged)
What does this MR do?
Part of gitlab-com/gl-infra/scalability#24 (closed)
Requires !18462 (merged)
This adds three new attributes to each sidekiq worker:
- Whether the job is latency sensitive
- Whether the job is CPU-bound, Memory-bound,
IO-bound, or unknown - Whether the job is dependent on external services in order for it to complete
Attribution
Sidekiq worker attribution is described in this spreadsheet:
https://docs.google.com/spreadsheets/d/1H9jpSKywxrguKR6otRAaIAHtztnjNeydUEM0jkMkg-c/edit
We use the following rules-of-thumb:
-
ProjectExportWorker
is the only memory-bound worker - All other jobs which spend more than 33% of their time on-cpu are categorized as CPU-bound
- All other jobs are
unknown
. Some of these may be io-bound, but whether or not they are does not change the way we treat them, so they all fall into a single category.
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation created/updated or follow-up review issue created -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Performance and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by Kamil Trzciński