Strategy in response to deprecation of Docker Machine by Docker
Status Update: 2022-10-11
We will maintain and provide full support for the GitLab Docker Machine fork until we have successfully developed and launched, as generally available, a replacement that supports autoscaling Runner on virtual machines hosted on the top three public cloud platforms (AWS, GCP, Azure).
We have started work on replacing Docker Machine. The new solution implements GitLab's next Runner Auto-scaling Architecture and will generalize the autoscaling system, which previously had been coupled with the Docker Machine executor.
Overview
Docker has ended active development and support for Docker Machine. A primary method for GitLab runner autoscaling on public cloud virtual machine instances is the Runner configured with the Docker Machine executor. The GitLab Runner Docker Machine executor, which uses Docker Machine, provisions and deletes virtual machines on multiple public cloud providers. As a result of Docker's EOL of Docker Machine, we are adjusting our plans and timelines related to replacing Docker Machine.
Summary of the plan to replace Docker Machine
- GitLab's next Runner Auto-scaling Architecture is the replacement for the current Docker Machine-based autoscaler.
- The implementation of the Next Runner Autoscaling blueprint to replace Docker Machine is a new component called Taskscaler. It uses a new library called Fleeting, which provides a “instance group” (IG) abstraction.
- These libraries are used in the GitLab Runner binary in a new “autoscaler” executor.
- Individual plugin binaries are created for each cloud provider via Hashicorp go-plugin.
- The plugin approach allows for new providers to be added without changing the Runner and also allows provider-specific plugins to be maintained separately from each other.
- This new implementation consolidates the autoscaling logic in one place (the Taskscaler), and simplifies the configuration of the underlying virtual machines VM's, as Fleeting only scales an instance group (IG) instead of a specific virtual machine instance type.