Skip to content

Update the Kubernetes executor's attach strategy to work with Windows pods

Arran Walker requested to merge 28081-windows-pods-attach-strategy into main

What does this MR do / Why was this MR needed?

  • Updates the attach strategy to work with Windows pods

    Previously we were executing a outer_script.ps1 inner_script.ps1 > logfile, but outside_script was also trying to write to logfile. This works on Linux, but not on Windows.

    The data being written was the exit code json information. I've changed it so that the exit code information is copied instead to stdout, and therefore is redirected to the logfile. I think the only change here is that anybody running kubectl logs manually to see the container logs will also see the json exit code line.

  • The log permissions init container has been updated to support Windows.

    I've renamed this so that it doesn't solely relate to the log file. For Windows, there's a similar permissions problem across the entire build root. On Linux, we solve this through using a umask. On Windows, we can solve it the same way as the log permission problem, using a command the sets permissions for future files as well as current files.

What's the best way to test this MR?

For Windows specifically:

  • Setup a Windows k8s cluster on GCP.

  • Authenticate locally with gcloud container clusters get-credentials <cluster name>

  • Configure runner's config.toml

    [[runners]]
    ...
    executor = "kubernetes"
    shell = "pwsh"
    
    [runners.feature_flags]
      # If deploying the Runner to Linux, but targetting Windows, this FF needs to be enabled for now
      FF_USE_POWERSHELL_PATH_RESOLVER = true
    
      # We explicitly want to ensure the new strategy is used, but that's the default anyway now
      # FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY = false
    [runners.kubernetes]
      image = "mcr.microsoft.com/powershell:lts-nanoserver-1809"
      [runners.kubernetes.node_selector]
        "kubernetes.io/arch" = "amd64"
        "kubernetes.io/os" = "windows"
        "node.kubernetes.io/windows-build" = "10.0.17763"
  • Run a simple job.

Linux

The json exit code going to stdout should continue to work. Running any job through the Kubernetes executor using the newer attach strategy should be a sufficient enough test.

What are the relevant issue numbers?

Closes #28081 (closed)

Edited by Arran Walker

Merge request reports