Roll out first routing rules using exclude_from_gitlab_com tag

We have some workers tagged (temporarily) with exclude_from_gitlab_com in gitlab-org/gitlab!61238 (merged). This means that we shouldn't listen to those queues in production. However, we should listen to them in staging: gitlab-com/gl-infra/k8s-workloads/gitlab-com!854 (merged)

Staging

This presents a good opportunity to test our worker routing rules in staging without having any production impact. We can set these rules in staging:

[
  ["tags=exclude_from_gitlab_com", "default"],
  ["*", null]
]

(We need to do this everywhere: the base role in chef, and for all rails containers in k8s.) NB: we use null for YAML (k8s) and also in JSON for chef, but this translates to nil in gitlab.rb

And then watch as those workers start running on the default queue, rather than their own named queues.

At this point, if everything works, we can also revert gitlab-com/gl-infra/k8s-workloads/gitlab-com!854 (merged): we don't need to listen to those queues in staging any more 🎉

Production

If this works well, we can roll the same configuration out to production. By default it will be a no-op, as none of those workers run in production, but we can test it from the console by doing this:

Chaos::SleepWorker.perform_async(0)

This will schedule a worker that does nothing (it sleeps for zero seconds). We should see that worker in the logs and metrics as happening on the default queue.