Detect Sidekiq worker changes through Danger
As an outcome of https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/349 discussions, it was noticed that one of the major concerns when handling background jobs infra is dealing with clogging high urgency jobs.
In the development side, it can happen when a new job is marked as urgent
, or an existing one goes from a low-urgency fleet to a high one. Given most of the time the SREs will only know of such event when an incident happens, here we propose adding a danger process to detect either new urgent workers or existing ones moving to a high urgency fleet.
To do this, we'll make danger compare the FOSS & EE versions all_queues.yml
of the source branch of a merge request with those of the target branch when there are changes to either of those files. When the comparison finds that selector attributes changed, it will add one of the teamScalability members as a reviewer in the comment it makes.
Based on review, the team member will involve and inform an SRE if needed.