Housekeeping of redundant Sidekiq queues when moving to routing rules

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Proposal

A Customer has noted that sidekiq queues are in need of housekeeping after moving to sidekiq routing rules https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#routing-rules

Customer feedback is

We've moved to sidekiq routing rules https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#routing-rules

We now have only 10 actual queues (default, mailers, etc..) and 200+ obsolete ones from lists: https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/workers/all_queues.yml

All those unused generate useless data for further analysis and metrics monitoring.

How can we remove all those queues, leaving only actual ones?

This last questions can be answered via https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1283#note_670668769 but this is only a short-term fix as when Sidekiq is restarted the obsolete queues return.

In issue https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1283 the need for this was considered for GitLab SAAS but it was decided it was not necessary.

• Can this housekeeping now be considered for self-managed installations as our customer move to sidekiq routing rules?

• Also, what is the current state of sidekiq in GitLab SAAS - do they still show the now obsolete queue?

Related issue from GitLab SAAS - https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1283