Skip to content

Revert Sidekiq default routing rules

Gregorius Marco requested to merge mg-revert-default-routing-rules into master

What does this MR do and why?

Reverting default Sidekiq routing rules due to performance regression on self-managed users. Full context: gitlab-com/gl-infra/scalability#1991 (closed).

In summary, we revert the default routing rules where we route all jobs to the default queue in 15.4. With this change, the default behavior goes back to named queue (1 queue per worker). Reverts !97908 (merged)

How to set up and validate locally

  1. Ensure routing_rules in config/gitlab.yml is an empty array (or empty):
    routing_rules: []
  2. Stop Sidekiq with gdk stop rails-background-jobs.
  3. Try to push a job in Rails console:
    [1] pry(main)> Settings.sidekiq.routing_rules
    => []
    [2] pry(main)> EmailReceiverWorker.queue
    => "email_receiver"
    [3] pry(main)> EmailReceiverWorker.perform_async
    => "1318845ec9a0594735a50faa"
  4. Check the job in Redis. It should go to the named queue with the same jid as above.
      ~/Doc/w/gdk-10-22/gitlab   mg-revert-default-routing-rules ❯ gdk redis-cli -n 1 lrange  resque:gitlab:queue:email_receiver 0 -1                minio 2022-07-15T03-44-22Z yarn 1.22.19  1.18.7  2.7.5
    1) "{\"retry\":3,\"queue\":\"email_receiver\",\"backtrace\":true,\"version\":0,\"args\":[],\"class\":\"EmailReceiverWorker\",\"jid\":\"1318845ec9a0594735a50faa\",\"created_at\":1667995255.0653179,\"meta.feature_category\":\"team_planning\",\"correlation_id\":\"f4239f653ba859aedc24dd53ea5b3d61\",\"worker_data_consistency\":\"always\",\"idempotency_key\":\"resque:gitlab:duplicate:email_receiver:e24f8243f3bdc71e54ba932255525eb20e3c5416e63890942eb16b1b38ed3f2f\",\"size_limiter\":\"validated\",\"enqueued_at\":1667995255.0890722}"

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Gregorius Marco

Merge request reports