Skip to content

Add support for Kubernetes runtime class

Georgi N. Georgiev requested to merge add-support-k8s-runtime-class into main

What does this MR do?

This MR is based on this community MR !1996 (closed)

Allows setting a runtime class for a specific kubernetes runner - https://kubernetes.io/docs/concepts/containers/runtime-class/

Why was this MR needed?

From original MR description:

For example, to make use of Kata Containers with Gitlab Runner for much better security.

Pod Annotations can be specified currently to use Kata Containers but it does not allow as much granularity, such as the ability to use kata-qemu-virtiofs for much better disk performance.

What's the best way to test this MR?

There are unit and integration tests. Running these should cover the functionality:

Unit tests:

 go test -v -run TestSetupBuildPod/RuntimeClass gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes

Integration tests:

go test -v -run TestRunTestsWithFeatureFlag/testKubernetesRuntimeClassFeatureFlag gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes

Manually:

Without specifying RuntimeClass

[[runners]]
  name = "Georgis-MacBook-Pro-2.local"
  url = "https://gitlab.com/"
  token = "<>"
  executor = "kubernetes"
  [runners.kubernetes]
    image = "alpine"
Running with gitlab-runner development version (HEAD)
  on Georgis-MacBook-Pro-2.local efc1edjz
Preparing the "kubernetes" executor
00:00
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine ...
Using attach strategy to execute scripts...
Preparing environment
Waiting for pod default/runner-efc1edjz-project-15339497-concurrent-069rm7 to be running, status is Pending
Waiting for pod default/runner-efc1edjz-project-15339497-concurrent-069rm7 to be running, status is Pending
Waiting for pod default/runner-efc1edjz-project-15339497-concurrent-069rm7 to be running, status is Pending
Running on runner-efc1edjz-project-15339497-concurrent-069rm7 via Georgis-MacBook-Pro-2.local...
Getting source from Git repository
00:03
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/ggeorgiev_/playground/.git/
Created fresh repository.
Checking out 1941ebbb as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ sleep 5000
❯ kubectl get pod runner-efc1edjz-project-15339497-concurrent-069rm7 -o yaml |grep runtimeClassName

With an invalid RuntimeClass:

[[runners]]
  name = "Georgis-MacBook-Pro-2.local"
  url = "https://gitlab.com/"
  token = "<>"
  executor = "kubernetes"
  [runners.kubernetes]
    image = "alpine"
    runtime_class_name = "invalid"
Running with gitlab-runner development version (HEAD)
  on Georgis-MacBook-Pro-2.local efc1edjz
Preparing the "kubernetes" executor
00:00
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine ...
Using attach strategy to execute scripts...
Preparing environment
00:00
ERROR: Job failed (system failure): prepare environment: setting up build pod: pods "runner-efc1edjz-project-15339497-concurrent-0" is forbidden: pod rejected: RuntimeClass "invalid" not found. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

With a valid class name:

Create it first:

apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: myclass
handler: docker
❯ pbpaste | kubectl create -f -
runtimeclass.node.k8s.io/myclass created

Verify:

Running with gitlab-runner development version (HEAD)
  on Georgis-MacBook-Pro-2.local efc1edjz
Preparing the "kubernetes" executor
00:00
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine ...
Using attach strategy to execute scripts...
Preparing environment
Waiting for pod default/runner-efc1edjz-project-15339497-concurrent-0mcm86 to be running, status is Pending
Waiting for pod default/runner-efc1edjz-project-15339497-concurrent-0mcm86 to be running, status is Pending
Waiting for pod default/runner-efc1edjz-project-15339497-concurrent-0mcm86 to be running, status is Pending
Running on runner-efc1edjz-project-15339497-concurrent-0mcm86 via Georgis-MacBook-Pro-2.local...
Getting source from Git repository
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/ggeorgiev_/playground/.git/
Created fresh repository.
Checking out 1941ebbb as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ sleep 5000
❯ kubectl get pod runner-efc1edjz-project-15339497-concurrent-0mcm86 -o yaml |grep runtimeClassName
  runtimeClassName: myclass

What are the relevant issue numbers?

Closes #26646 (closed)

Edited by Georgi N. Georgiev

Merge request reports