Expose runner limit error message on registration
What does this MR do?
This MR adds additional logic to decode and log the message present in the body of the /api/v4/runners
response in the case where a Bad Request (400)
status code is returned.
Why was this MR needed?
As noted in #27840, when we enabled the :ci_runner_limits
FF globally some users started hitting the limits when registering a specific type of runner and were left wondering what had happened since the error message reported by gitlab-runner register
is too generic (it just reported that "you may be having network problems"). When a plan limit is hit on GitLab, it returns an error in the body and a Bad Request (400)
status code:
Runner type | HTTP response body |
---|---|
group | {"message":{"runner_namespaces.base":["Maximum number of ci registered group runners (50) exceeded"]}} |
project | {"message":{"runner_projects.base":["Maximum number of ci registered project runners (50) exceeded"]}} |
In this MR, we now treat the 400
error in a special way and try to decode the JSON payload. If we find any of the expected fields, we use that in the log message in the status
field. For that, I needed to slightly alter network/client.go
so that it accepts more than one status code to decide when to decode the JSON response. I also created a GitLabClient.doJSONMulti
for the same purpose (which is now called by the previous GitLabClient.doJSON
for compatibility).
What's the best way to test this MR?
-
On GitLab.com, create a new group named
<your-user-name>-test-runner-limits
. -
Enable the
:ci_runner_limits
FF for that group. For that, go to #production and type:/chatops run feature set --group=<your-user-name>-test-runner-limits ci_runner_limits true
-
Build this branch:
git switch pedropombeiro/27840/expose-runner-limit-error-message && make runner-bin-host
-
Register 51 runners on the new group:
Assuming an environment variable
GROUP_RUNNERS_REGISTRATION_TOKEN
set to the runners registration token in your group's (https://gitlab.com/groups/your-username-test-runner-limits/-/settings/ci_cd) Runner registration page:for i in $(seq 1 51); do \ out/binaries/gitlab-runner register -config /tmp/config.test.toml \ --non-interactive \ --executor "shell" \ --url "https://gitlab.com/" \ --description "group test runner" \ --tag-list "shell,gdk,mac,test" \ --run-untagged="false" \ --locked="false" \ --access-level="not_protected" \ --registration-token="$GROUP_RUNNERS_REGISTRATION_TOKEN"; \ done
Upon registering the 51st runner (assuming your account is on the free plan) you should see the following error message:
-
On GitLab.com, create a new project named
test-project-runner-limits
under the group you created earlier. -
Enable the
:ci_runner_limits
FF for that project. For that, go to #production and type:/chatops run feature set --project=<your-user-name>-test-runner-limits/test-project-runner-limits ci_runner_limits true
-
Register 51 runners on the new project:
Assuming an environment variable
PROJECT_RUNNERS_REGISTRATION_TOKEN
set to the runners registration token in your project's (https://gitlab.com/your-username-test-runner-limits/test-project-runner-limits/-/settings/ci_cd) Runner registration page:for i in $(seq 1 51); do \ out/binaries/gitlab-runner register -config /tmp/config.test.toml \ --non-interactive \ --executor "shell" \ --url "https://gitlab.com/" \ --description "group test runner" \ --tag-list "shell,gdk,mac,test" \ --run-untagged="false" \ --locked="false" \ --access-level="not_protected" \ --registration-token="$PROJECT_RUNNERS_REGISTRATION_TOKEN"; \ done
Upon registering the 51st runner (assuming your account is on the free plan) you should see the following error message:
-
Unregister all runners from this MR:
out/binaries/gitlab-runner unregister --all-runners --config /tmp/config.test.toml
What are the relevant issue numbers?
Closes #27840