Windows Shared Runners MVC

👋 promoted to epic &1768 (closed) 👋

Problem statement

Currently we do not offer shared runners on GitLab.com for Windows. This is primarily because Docker Machine doesn't support Windows (#4338 (closed)), and we don't yet support Windows nodes in Kubernetes (#4014 (closed)).

Most of our competition, like Circle CI and GitHub, already have support. This is a major gap for Windows developers, as they have to set up their own runners in order to use CI/CD.

Goal

The goal is the smallest iteration which enables us to offer Windows shared runners on GitLab.com.

Requirements

We should keep the requirements to the absolute minimum to run customer jobs:

• Security: User data should be isolated between jobs
• Windows license compliance
• Scaling: Some method of tuning the runner pool

What are not requirements

It is also important to note what is not a requirement for the MVC:

• Windows specific metering/billing
• Full autoscaling
• Features like customizable "warm" nodes, etc.
• Simple, easy setup/maintenance/upgrade
• All existing Runner features (e.g. Services)
• Customer supplied containers/VM's. We can standardize on a stock VM image/container, like Circle CI. This can reduce the complexities of compatible version matrixes, and long download/start up times. (VM's and full containers can be ~5GB)

WIP Proposal

• Select the boring solution, which is launching VM's using a tool (terraform, cloud provider CLI). This can be an extension of our custom executor, or the basis of something new.
• Determine whether we should utilize plain VM's or Docker containers. Note that there are limitations with Windows containers right now, so it may be best to go with a VM, but if containers are easier we can start there.
• Tear down VM after job completes, to ensure security/privacy.
• Windows runners will be configured to only run tagged jobs, and look for jobs with windows tag

Rationale

• Windows nodes in Kubernetes requires version 1.14, which no managed k8s provider has GA'd yet.
• Infrakit/Deploykit is essentially abandoned
• Leaves ansible/terraform/shell scripting
  • This could form the basis of an easier to use Custom Executor, for example if the script spins up a VM and returns a Docker IP/port to the main runner process, which then would proceed as normal.