11. Enable the runner-managers on the XL shard
Prepare
-
Increase the concurrency to 750
👉 https://gitlab.com/gitlab-com/gl-infra/chef-repo/-/blob/master/roles/runners-manager-saas-linux-xlarge-amd64.json?ref_type=heads#L19 -
Mention @sre-oncall
in #production with a FYI message, letting them know you're about to enable a new runner's shard, and if timing is OK for that. -
Monitor https://dashboards.gitlab.net/d/ci-runners-deployment/ci-runners-deployment-overview?orgId=1.
Execute
-
/runner run start saas-linux-xlarge-amd64 green
in #2635 (closed)
Verify
-
Verify the new shard xlarge
is starting to report traffic in the Grafana dashboard above☝ -
Look at the logs to verify jobs are getting picked 👉 https://log.gprd.gitlab.net/app/r/s/DVD1w -
Ensure the Apdex is healthy with no newly reported errors 👉 https://dashboards.gitlab.net/goto/tqXJayj4R?orgId=1
If things go south
-
Verify there's no errors, if any, stop the fleet: /runner run stop saas-linux-xlarge-amd64 green
Edited by Davis Bickford