wkld-detect-unplanned-node-rolling-updates is executed on the wrong runner tags

On capo CI, we use the runner max parallelism to limit how many clusters are deployed on our test infra. We use a special job to keep the runner busy during the testing phase ( so that other CIs cannot start on the runner slot ).

We have two kinds of runners:

  • deploy: for all deploy job, those runners should be kept busy to prevent other clusters from being deployed (limited to 8 total jobs)
  • test / misc to run other tasks ( limited to 16 total jobs, but they are usually rather short )

On deployments with a workload cluster and an update to the mgmt cluster, we have the following dependency chain between our jobs:

deploy-mangement-cluster -> deploy-workload-cluster -> update-management-cluster -> wkld-detect-unplanned-node-rolling-updates -> update-workload-cluster -> keep runner busy until all tests are done.

all those job run on the deploy runner, except for wkld-detect-unplanned-node-rolling-updates. The deploy runner is "freed" during this job, making it possible for other CIs to start a deployment.

This can lead to resource exhaustion on our test infra and make the current deployment stuck waiting for the runner.