Skip to content

Remove dual-namespace polling support in Sidekiq

Sylvester Chin requested to merge sc1-remove-dual-poll-support into master

What does this MR do and why?

This MR deprecates the dual-namespace poller introduced in 16.4 to migrate sidekiq-related keys from namespaced to non-namespaced keys.

In 16.7, we can drop the dual-namespace polling since it is no longer needed.

See gitlab-com/gl-infra/scalability#2288 (closed)

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Run gdk on master commit

  1. Run monitor on the Redis db for queueing
gdk redis-cli -n 1 monitor
  1. Observe dual-namespace polling. For example:
1699505637.842850 [1 unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket] "brpop" "resque:gitlab:queue:project_import_schedule" "queue:project_import_schedule" "queue:mailers" "resque:gitlab:queue:mailers" "resque:gitlab:queue:hashed_storage:hashed_storage_migrator" "resque:gitlab:queue:default" "queue:hashed_storage:hashed_storage_project_rollback" "resque:gitlab:queue:hashed_storage:hashed_storage_project_migrate" "queue:service_desk_email_receiver" "queue:email_receiver" "queue:hashed_storage:hashed_storage_migrator" "queue:default" "queue:hashed_storage:hashed_storage_project_migrate" "queue:hashed_storage:hashed_storage_rollbacker" "resque:gitlab:queue:service_desk_email_receiver" "resque:gitlab:queue:email_receiver" "resque:gitlab:queue:hashed_storage:hashed_storage_rollbacker" "resque:gitlab:queue:hashed_storage:hashed_storage_project_rollback" "2"

  1. Run gdk restart on this branch

  2. Repeat step 1. Observer that no namespaced queues are in the brpop command now.

1699505730.162376 [1 unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket] "brpop" "queue:default" "queue:mailers" "queue:email_receiver" "queue:hashed_storage:hashed_storage_rollbacker" "queue:service_desk_email_receiver" "queue:project_import_schedule" "queue:hashed_storage:hashed_storage_migrator" "queue:hashed_storage:hashed_storage_project_rollback" "queue:hashed_storage:hashed_storage_project_migrate" "2"

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sylvester Chin

Merge request reports