Pilot runners: Fix GitLab Duo in pilot-enabled projects
Our strategy of deploying pilot runners was to not change anything in the original project's pipelines meaning jobs that were running on gitlab-org tags would continue to do so and our runners will be tagged as such. For this to happen deterministically we had to turn off shared and instance runners for these projects. This had the unfortunate side effect that gitlab due does not work 1. It needs instance runners with gitlab--duo tag 2. If we tag our runners with this tag we still get an error ![image](/uploads/668867939cd710ed3b1d5329d0a2d1df/image.png){width=730 height=148} Strategy ---- We will add an env variable toggleable switch for all the jobs where by default we'll run on `functions-*` runners but if the need arises jobs will be able to go back to their default tags at which point the shared and instance runners will pick them up. ## Implementation Plan ### Solution Single `USE_PILOT_RUNNERS` toggle (default: `true`) in `gitlab-runner` CI configuration that makes all runner tags variable-driven. When disabled, jobs fall back to shared/instance runner tags, restoring GitLab Duo. ### Files Changed (in `gitlab-org/gitlab-runner`) **1. `.gitlab/ci/_project_canonical.gitlab-ci.yml`** — Tag anchors use CI variable references: ```yaml .instance-default: $RUNNER_TAG_DEFAULT .instance-default-docker: $RUNNER_TAG_DOCKER .instance-2xlarge: $RUNNER_TAG_2XLARGE .instance-medium: $RUNNER_TAG_MEDIUM .instance-windows-2019: $RUNNER_TAG_WINDOWS_2019 .instance-windows-2022: $RUNNER_TAG_WINDOWS_2022 ``` **2. `.gitlab/ci/_common.gitlab-ci.yml`** — Added toggle and pilot tag defaults: ```yaml USE_PILOT_RUNNERS: "true" RUNNER_TAG_DEFAULT: "functions-pilot-linux-amd64" RUNNER_TAG_DOCKER: "functions-pilot-linux-amd64" RUNNER_TAG_2XLARGE: "functions-pilot-linux-amd64" RUNNER_TAG_MEDIUM: "functions-pilot-linux-amd64" RUNNER_TAG_WINDOWS_2019: "functions-pilot-windows-amd64" RUNNER_TAG_WINDOWS_2022: "functions-pilot-windows-amd64" ``` **3. `.gitlab/ci/_rules.gitlab-ci.yml`** — Workflow rule overrides tags when toggle is off: ```yaml - if: '$USE_PILOT_RUNNERS == "false"' variables: RUNNER_TAG_DEFAULT: "gitlab-org" RUNNER_TAG_DOCKER: "gitlab-org-docker" RUNNER_TAG_2XLARGE: "saas-linux-2xlarge-amd64" RUNNER_TAG_MEDIUM: "saas-linux-medium-amd64" RUNNER_TAG_WINDOWS_2019: "windows-1809" RUNNER_TAG_WINDOWS_2022: "windows-21h1" ``` **4. `.gitlab/ci/build.gitlab-ci.yml`** — Fixed hardcoded tag: ```yaml # before helper images: tags: - saas-linux-2xlarge-amd64 # after helper images: tags: - !reference [.instance-2xlarge] ``` ### Tag Mapping | Variable | Pilot (default) | Fallback (`USE_PILOT_RUNNERS=false`) | |---|---|---| | `RUNNER_TAG_DEFAULT` | `functions-pilot-linux-amd64` | `gitlab-org` | | `RUNNER_TAG_DOCKER` | `functions-pilot-linux-amd64` | `gitlab-org-docker` | | `RUNNER_TAG_2XLARGE` | `functions-pilot-linux-amd64` | `saas-linux-2xlarge-amd64` | | `RUNNER_TAG_MEDIUM` | `functions-pilot-linux-amd64` | `saas-linux-medium-amd64` | | `RUNNER_TAG_WINDOWS_2019` | `functions-pilot-windows-amd64` | `windows-1809` | | `RUNNER_TAG_WINDOWS_2022` | `functions-pilot-windows-amd64` | `windows-21h1` |
issue