Skip to content

Attribute all Sidekiq workers according to their workloads

Andrew Newdigate requested to merge an-mark-jobs-as-latency-sensitive into master

Merge sequence: !18462 (merged) 🚋 !18066 (merged) 🚋 !18877 (merged)

What does this MR do?

Part of gitlab-com/gl-infra/scalability#24 (closed)

Requires !18462 (merged)


This adds three new attributes to each sidekiq worker:

  1. Whether the job is latency sensitive
  2. Whether the job is CPU-bound, Memory-bound, IO-bound, or unknown
  3. Whether the job is dependent on external services in order for it to complete

Attribution

Sidekiq worker attribution is described in this spreadsheet:

https://docs.google.com/spreadsheets/d/1H9jpSKywxrguKR6otRAaIAHtztnjNeydUEM0jkMkg-c/edit

We use the following rules-of-thumb:

  • ProjectExportWorker is the only memory-bound worker
  • All other jobs which spend more than 33% of their time on-cpu are categorized as CPU-bound
  • All other jobs are unknown. Some of these may be io-bound, but whether or not they are does not change the way we treat them, so they all fall into a single category.

Does this MR meet the acceptance criteria?

Conformity

Performance and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Kamil Trzciński

Merge request reports