Gitlab runner should support specifying pods via config.toml file

Status Update (2023-03-22)

You can now apply this configuration using the custom podspec feature. The feature is currently behind a feature flag FF_USE_ADVANCED_POD_SPEC_CONFIGURATION

Config Example

[[runners]]
  environment = ["FF_USE_ADVANCED_POD_SPEC_CONFIGURATION=true"]
  [runners.kubernetes]
    image = "alpine"
    [[runners.kubernetes.pod_spec]]
      name = "tester"
      patch_type = "strategic"
      patch = '''
      initContainers:
        - name: "init-permissions"
          resources:
            limits:
              cpu: "..."
            requests:
              cpu: "..."
      '''

We are also capturing any feedback wrt this feature in the issue Overwrite generated Kubernetes pod specificatio... (#29659 - closed)

Documentation

---

It is unpractical to expose every possible option for executor pod (resource limits, names, securityContext, mounts, etc), gitlab runner should simple allow providing full POD spec within config.toml. It would allow full flexibility of executor configuration without ongoing maintenance burden of exposing more and more fields via config.toml. For instance it would become trivial to configure different security policies for different containers.

runner should be constructing Pod spec to be created in Kubernetes using executorPodSpec as a base and overlaying any job specific parameters such as "image", dynamically generated imagePullSecrets, service account, etc on top of it.

Configuration example how PodSpec might look like:

[runners.kubernetes.executorPodSpec]
apiVersion = "v1"
kind = "Pod"

[metadata]

  [metadata.labels]
  label1  = value1

[spec]

  [[spec.containers]]
  name = builder
  command = [
    "/bin/bash",
    "/scripts/entrypoint"
  ]
  ...

  [[spec.containers.securityContext]]
  privileged = true

If executorPodSpec is present together with any simple configuration options such as "pod_labels", "image", it should be considered configuration error to avoid confusion. Any overwrites configured in job spec should be applied as usual.

Array of containers should have following semantics:

• any services which has same names as already predefined ones in executorPodSpec should be rejected and job error returned

I am more than happy to work in MR if there is an interest from Gitlab dev in merging such feature.