Skip to content

Add support for determining helper image from node selector information

Arran Walker requested to merge ajwalker/k8s-windows into main

⚠ targets and depends on !2836 (merged) ⚠

What does this MR do?

  • Determines which helper image to use as early as possible, during Prepare(), so that the correct setup can be done according to the OS the build will be deployed to.
  • Uses provided node selector information to determine the best helper image to select. Whilst this MR has a focus on Windows, the same logic should apply for targetting ARM nodes, for example, in a mixed cluster.

Why was this MR needed?

Users can already provide node selectors to target specific nodes/operating systems, but the helper image selected is always based off of where Runner is hosted, not necessarily where the build will be deployed. This uses the same node selector logic to determine the best image to use.

In a follow-up, we can also support RuntimeClass resources, once we support them.

What's the best way to test this MR?

I don't think we have a way to test integrations across different platforms for Kubernetes.

Manual QA:

  • Setup a Windows k8s cluster on GCP.
  • Authenticate locally with gcloud container clusters get-credentials <cluster name>
  • Configure runner's config.toml 
    [[runners]]
  url = "https://gitlab.com/"
  token = "..."
  executor = "kubernetes"
  shell = "pwsh"
  [runners.feature_flags]
    # If deploying the Runner to Linux, but targetting Windows, this FF needs to be enabled for now
    FF_USE_POWERSHELL_PATH_RESOLVER = true
  [runners.kubernetes]
    ...
    image = "mcr.microsoft.com/powershell:lts-nanoserver-1909"
    # helper_image = "gitlab/gitlab-runner-helper:x86_64-07807de0-servercore1909"
    [runners.kubernetes.node_selector]
      "kubernetes.io/arch" = "amd64"
      "kubernetes.io/os" = "windows"
      "node.kubernetes.io/windows-build" = "10.0.18363"

Execute a simple job.

The debug log will successfully print: "Using helper image: gitlab/gitlab-runner-helper:x86_64-latest-servercore1909"

Unfortunately, this will fail with the confusing error message container not found, despite this image being found. We have a problem with our x86_64-latest-servercore1909 tag - it hasn't been updated in year...

Forcing the helper image by uncommenting # helper_image will work however. Despite now expliticly providing the image, the node selector is still useful as now Runner knows which OS the build is being executed on (which does apply additional configuration: default path names, security context etc).

What are the relevant issue numbers?

closes #4014 (closed)

Edited by Arran Walker

Merge request reports