Part of https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/15449
Depends on gitlab-com/runbooks!4557 (merged)