2025-09-29: Self-managed GitLab Runners receiving 401 unauthorized when picking up CI jobs
Self-managed GitLab Runners receiving 401 unauthorized when picking up CI jobs (Severity 2 (High))
It seems likely that this recent MR (gitlab-org/gitlab!206026 (merged)) deployed to GitLab.com caused a regression in which some older runners would get a 401 when polling for jobs. This meant that certain customers with their own runners would have stuck CI jobs with no runner that can pick them up.
The main resolution was to rollback our deployment.
The other resolution was to revert the MR in master in gitlab-org/gitlab!206642 (merged) . The unblocked future deploys.
Then we did another deploy to production with the revert.
The overall impact to users was that for a period of time customers with their own runners could not pick up jobs on GitLab.com . As such they had stuck CI pipelines until the deploy was reverted.
This ticket was created to track INC-4255, by incident.io