Notify administrators if runners are outdated/invalid
Problem to solve
Customer discovered an issue where older runners were constantly spamming gitlab on old API endpoints (57,000 times per day).
10.0.0.110 - - [20/Feb/2019:07:25:58 -0500] "POST /ci/api/v1/builds/register.json HTTP/1.1" 404 34 "-" "gitlab-ci-multi-runner 1.8.1 (1-8-stable; go1.7.3; windows/386)" servertime=0.010;
10.0.0.112 - - [19/Feb/2019:13:37:15 -0500] "POST /ci/api/v1/builds/register.json HTTP/1.1" 404 34 "-" "gitlab-ci-multi-runner 1.9.2 (; go1.7.4; linux/386)" servertime=0.012;
10.0.0.33 - - [19/Feb/2019:17:02:43 -0500] "POST /ci/api/v1/builds/register.json HTTP/1.1" 404 34 "-" "gitlab-ci-multi-runner 1.8.1 (1-8-stable; go1.7.3; linux/amd64)" servertime=0.010;
10.0.0.43 - - [19/Feb/2019:15:23:25 -0500] "POST /ci/api/v1/builds/register.json HTTP/1.1" 404 34 "-" "gitlab-ci-multi-runner 1.3.3 (1-3-stable; go1.6.2; linux/amd64)" servertime=0.010;
Proposal
Provide email/report of details that can be actionable by an administrator
- IP address (arguably the most important)
- Flapping start date
- Runner version/OS
Other information gleaned from the process:
- Number of hits; even an average per hour/day would be helpful as a metric
- DNS lookup of IP address (if available)
Potential other operations that would be nice for the admin:
- A configurable threshold for auto-blacklisting a flapping runner
- UI for managing the blacklisting (manual block/unblock)
Links / references
Context on Internal ZD