Set up full routing rules in staging and production
In working towards #989 (closed), we can set up our full current shard configuration in staging and production, but with null
on the right-hand side of most selectors so they are no-ops.
Because evaluation of these rules stops at the first match, we don't need to concatenate selectors or use negation, so this should end up simpler than what we have currently.
This would look something like:
[
["name=project_import_schedule", null], # we cannot migrate this worker yet, https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1087
["resource_boundary=cpu&urgency=high&tags!=requires_disk_io", null], # urgent-cpu-bound
["resource_boundary=memory", null], # memory-bound
["feature_category=global_search&urgency=throttled", null], # elasticsearch
["resource_boundary!=cpu&urgency=high", null], # urgent-other
["resource_boundary=cpu&urgency=default,low", null], # low-urgency-cpu-bound
["feature_category=database&urgency=throttled", null], # database-throttled
["feature_category=gitaly&urgency=throttled", null], # gitaly-throttled
["tags=exclude_from_kubernetes", null], # catchall on VMs
["tags=exclude_from_gitlab_com", null|default], # default if https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1072 is done first, 'null' if it is not.
["*", null] # catchall on k8s
]
NB: we use null
for YAML (k8s) and also in JSON for chef, but translates to nil in gitlab.rb
We also need to:
-
Optionally, if delivery#1738 (closed) hasn't happened yet: Make catchnfs VMs listen to the (non-existent)No longer required; 1738 happend.catchnfs
queue name. We can do this by passingcatchnfs
as the queue list for it and disabling the queue selector. -
Add name=default
to https://ops.gitlab.net/gitlab-cookbooks/chef-repo/-/blob/master/tools/sidekiq-config/sidekiq-queue-configurations.libsonnet to stop catchall VMs processing thedefault
queue.
Edited by Craig Miskell