Skip to content

Register Sidekiq queues only from routing rules

Gregorius Marco requested to merge mg-sidekiq-version-queues into master

What does this MR do and why?

Register Sidekiq queues only from routing rules

Also clean up other queues that are outside routing rules. Most instances would have hundreds of unused queues back when we were still running queues per worker.

Changelog: changed

For #415298

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

With empty routing rules

  1. Ensure sidekiq.routing_rules is empty in gitlab.yml

  2. Restart sidekiq gdk restart rails-background-jobs

  3. Check Rails console

    [8] pry(main)> Sidekiq::Queue.all.map(&:name)
    => ["default", "mailers"]
  4. Or check queues set in Redis

❯ gdk redis-cli -n 1 smembers queues
1) "default"
2) "mailers"

With a custom routing rules

  1. Paste this in gitlab.yml

      sidekiq:
        log_format: json # (default is also supported)
        routing_rules:
          - ["urgency=high", "urgent"]
          - ["urgency=low", "not-urgent"]
          - ["*", "default"]
  2. Restart gdk restart rails-background-jobs

  3. Check queues

    ❯ gdk redis-cli -n 1 smembers queues
    1) "urgent"
    2) "not-urgent"
    3) "default"
    4) "mailers"
Edited by Gregorius Marco

Merge request reports