## Problem statement Currently we do not offer shared runners on GitLab.com for Windows. This is primarily because Docker Machine doesn't support Windows (https://gitlab.com/gitlab-org/gitlab-runner/issues/4338), and we don't yet support Windows nodes in Kubernetes (https://gitlab.com/gitlab-org/gitlab-runner/issues/4014). Most of our competition, like Circle CI and GitHub, already have support. This is a major gap for Windows developers, as they have to set up their own runners in order to use CI/CD. ## Goal The goal is the smallest iteration which enables us to offer Windows shared runners on GitLab.com. ## Requirements We should keep the requirements to the absolute minimum to run customer jobs: * Security: User data should be isolated between jobs * Windows license compliance * Scaling: Some method of tuning the runner pool #### What are not requirements It is also important to note what is not a requirement for the MVC: - Windows specific metering/billing - Full autoscaling - Features like customizable "warm" nodes, etc. - Simple, easy setup/maintenance/upgrade - All existing Runner features (e.g. Services) - Customer supplied containers/VM's. We can standardize on a stock VM image/container, like Circle CI. This can reduce the complexities of compatible version matrixes, and long download/start up times. (VM's and full containers can be ~5GB) ## Proposal - Create a new plugin for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html) - This plugin will create/run/destroy a VM per job. - This will be a "fire and forget" method where we will not keep track of any state to keep thing simple. This, of course, has limitation - These VMs are going to be hosted on Google Cloud Platform - We will use our own custom VM that will have docker installed an all the dependencies [GitLab Runner requires](https://docs.gitlab.com/runner/executors/custom.html#prerequisite-software-for-running-a-job) - Windows runners will be configured to only run tagged jobs, and look for jobs with `windows` tag - Machine pooling would not be considered as part of the MVC. ### Task breakdown Board: https://gitlab.com/gitlab-org/gitlab-runner/-/boards/1252488?label_name[]=Shared%20Runners%3A%3AWindows An issue for each task will be created for it to be split between engineers but below is the overall task breakdown of the MVC - 1 binary with 3 purposes - 1 to prepare - Spin up everything - Store information about machines in a file, In case of a problem can investigate and cleanup - 1 for run - Take script created by GitLab Runner an execute to on the VM - 1 for cleanup - Delete resource group that is specified - Task breakdown - Handle all 3 stages (prep/run/cleanup) - Add CLI - Add saving the state to a file / configuration - Prepare the image for VMs - Installed with the following software - Docker - Git/ Git LFS - gitlab-runner - Pre build images - Task breakdown - Running packer project - Install git, git-lfs, gitlab-runner etc. - Azure exporter for Prometheus - The current number of created instances - Quota usage (may be in future iteration) - Following the idea of https://gitlab.com/gitlab-org/ci-cd/gcp-exporter - Runner Managers with the proper configuration - Define firewall rules (terraform) - Updates for cookbook-gitlab-runner - Preparing the infrastructure and configuration - Prepare DNS names - Prepare chef role: - Provision the custom executor binary - Provision the configuration for it - Pause GitLab Runner custom machines and move to the autoscaled ones - Create new label ~”Shared Runners::Windows” so we can track all issues related to this ### Test cases To validate and test this Runner we are going to be using https://gitlab.com/gitlab-org/gitlab-runner project which runs windows test and builds a docker image for every pipeline. With this, we can test the scaleability of the executor. ## Useful links - [Windows Shared Runner readiness](https://gitlab.com/gitlab-com/gl-infra/readiness/merge_requests/12) - [Windows Shared Runner blog post](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/34599) - [Launch Windows Runner beta](https://gitlab.com/gitlab-org/gitlab-runner/issues/4815) - [Windows CI infrastructure](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/126)