Skip to content

Add worker_class argument to Sidekiq queues APIs

Sean McGivern requested to merge add-worker-parameter-to-queues-api into master

What does this MR do and why?

These APIs allow you to delete Sidekiq jobs from a queue. We now support having multiple worker classes use the same queue, so we need an extra argument to support that case - otherwise it would be possible to delete jobs from a less-problematic worker when just meaning to target a particular worker.

For gitlab-com/gl-infra/scalability#1087 (closed).

How to set up and validate locally

This is easiest if Sidekiq isn't running (i.e. accepting jobs), because then the list of jobs in a queue will be stable: gdk stop rails-background-jobs.

Then add this under sidekiq in config/gitlab.yml:

routing_rules: [['*', 'default']]

Start a console (bundle exec rails c) to populate some jobs:

AuthorizedProjectsWorker.perform_async(1)
AuthorizedProjectsWorker.perform_async(2)
MergeWorker.perform_async(3)
Sidekiq::Queue.new('default').map(&:klass)
#=> ["MergeWorker", "AuthorizedProjectsWorker", "AuthorizedProjectsWorker"]

Then use this API to delete the AuthorizedProjectsWorker jobs:

$ curl -s -H "Private-Token: $GITLAB_API_TOKEN_LOCAL" -X DELETE 'http://localhost:3000/api/v4/admin/sidekiq/queues/default?worker_class=AuthorizedProjectsWorker' | jq
{
  "completed": true,
  "deleted_jobs": 2,
  "queue_size": 1
}

Which we can validate in the console:

Sidekiq::Queue.new('default').map(&:klass)
#=> ["MergeWorker"]
Edited by Sean McGivern

Merge request reports