Skip to content

Implement Sidekiq pause control middleware

Dmitry Gruzd requested to merge 389754-pause-workers-middleware into master

What does this MR do and why?

This MR implements pause control middleware to be able to pause Sidekiq workers conditionally. Paused jobs will be stored in a separate ZSET until the pause strategy becomes inactive. When that happens, PauseControlWorker puts the jobs back into the queues.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Launch gdk tail -f rails-background-jobs in a terminal window
  2. Execute Zoekt::IndexerWorker.perform_async(7)
  3. Ensure that the job was processed with "job_status":"done"
  4. Run ::Feature.enable(:zoekt_pause_indexing)
  5. Execute Zoekt::IndexerWorker.perform_async(7) once again
  6. Ensure that it returns nil
  7. Ensure that Gitlab::SidekiqMiddleware::PauseControl::PauseControlService.queue_size('Zoekt::IndexerWorker') returns 1
  8. Run ::Feature.disable(:zoekt_pause_indexing)
  9. After running PauseControl::ResumeWorker.new.perform you should see that the paused job was processed

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #389754 (closed)

Edited by Dmitry Gruzd

Merge request reports