Allow to interrupt running sidekiq jobs
What does this MR do?
This is Proof-of-Concept. It is concurrent-safe. It is cheap.
This implements the proposal from: https://gitlab.com/gitlab-org/gitlab-ce/issues/51096#note_203681105:
- This adds a middleware to track all threads for running jobs.
- This makes sidekiq to watch for redis-delivered notifications.
- This makes be able to send notification to interrupt running sidekiq jobs.
- This does not take into account any native code, as
Thread.raise
generates exception once the control gets back to Ruby. - The separate measure should be taken to interrupt gRPC, shellouts, or anything else that escapes Ruby.
How to use it?
jid = Chaos::SleepWorker.perform_async(1000);
sleep 2;
Gitlab::SidekiqStatus::CancelJobs.cancel_job(jid);
Remarks
We don't need any new code to remove jobs from Sidekiq, as this is already provided by Sidekiq::API:
queue = Sidekiq::Queue.new("mailer")
queue.each do |job|
job.klass # => 'MyWorker'
job.args # => [1, 2, 3]
job.delete if job.jid == 'abcdef1234567890'
end
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry for user-facing changes, or community contribution. Check the link for other scenarios. -
Documentation created/updated or follow-up review issue created -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Performance and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by 🤖 GitLab Bot 🤖