Migrate all runners to GitLab Runner auto-scaling
We are using GitLab Runner auto-scaling for GitLab.com gitlab-ce and gitlab-ee with great success. Scaling up to over 100 machines.
When we release GitLab Runner 1.1 (next week) we should start using the new release to enable auto-scaling on Shared Runners. Currently we use a fixed numbers of GitLab Runners (public-runner-*.gitlap.com
[managed by Ansible]).
Ideally we should switch to the docker-ci-1.gitlap.com
and docker-ci-2.gitlap.com
to manage all our runners infrastructure. This two machines will have two instances of GitLab Runner, the GitLab Runner will manage the runners infrastructure with docker-machine.
Currently only the docker-ci-1.gitlap.com
is configured. It is providing runners to https://dev.gitlab.org (up to 20 machines) and to https://gitlab.com (mentioned repositories) (up to 140 machines). We have docker-ci-2.gitlap.com
ready to be configured with mirror configuration as docker-ci-1.gitlap.com, to have HA setup.
The GitLab Runner for docker-ci-1.gitlap.com
configuration can be viewed here: sudo cat /etc/gitlab-runner/config.toml
/.
Machines on docker-ci-1.gitlap.com are managed with docker-machine
.
I did configure two extra machines for caching the runner images and build caches:
-
gitlab-runner-cache-server
- it's running minio, the S3-compatible file server, -
gitlab-runner-registry-mirror
- it's running Docker Registry and is used as a proxy in front of Docker Hub,
The dynamically provisioned build machines use the private networking (of Digital Ocean) to connect to cache server and registry mirror.
More about auto-scaling: Implementation: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/53 User-guide: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/95
cc @pcarranza