Loading
Add hooks for deferred and resumed concurrency limit jobs
What does this MR do and why?
Adds two optional class methods that a worker with a concurrency limit can define:
on_concurrency_limit_deferred(args): called when a job is deferred.on_concurrency_limit_resumed(args): called when a deferred job is resumed.
A job can depend on state that expires while it waits in the concurrency limit queue. These hooks let the worker keep that state alive. The first planned user is CreatePipelineWorker, which needs to keep its pipeline creation request alive while it waits under a per-user limit.
Details
- The hooks receive the job arguments, uncompressed when the size limiter compressed them. The stored job stays compressed.
- Errors in a hook are tracked with
Gitlab::ErrorTrackingand not raised, so a hook cannot lose a deferred job or stop a resume. - The hooks work with any concurrency limit, with or without a scope.
deferredis called fromMiddleware#defer_job!.resumedis called fromQueueManager#prepare_and_store_metadata, before the job is pushed back to Sidekiq.
References
Problem: https://gitlab.com/gitlab-org/gitlab/-/work_items/604640
| Step | Merge request | What it does | Needs |
|---|---|---|---|
| 1 | !264227 (merged) (merged) | Adds concurrency_limit_scope, a per-namespace or per-user share of a worker's concurrency limit. |
- |
| 2 | This MR | Adds the on_concurrency_limit_deferred and on_concurrency_limit_resumed worker hooks. |
1 |
| 3 | !264410 (merged) | Uses the per-user scope for CreatePipelineWorker and MergeRequests::CreatePipelineWorker. |
1 |
| 4 | !267879 (merged) | Keeps a pipeline creation request in progress while its job is deferred, using the hooks from step 2. | 2 |
Steps 3 and 4 do not depend on each other and can merge in any order.
How to set up and validate locally
class TestLimitedWorker
include ApplicationWorker
concurrency_limit -> { 1 }
def self.on_concurrency_limit_deferred(args) = Rails.logger.info("deferred #{args}")
def self.on_concurrency_limit_resumed(args) = Rails.logger.info("resumed #{args}")
def perform(*) = sleep 30
end- Enable the
sidekiq_concurrency_limit_middlewareops flag. - Enqueue
TestLimitedWorker3 times. - Check the log: 2
deferredlines, then aresumedline each time a job finishes.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.
Edited by Furkan Ayhan