Make directories for scripts and logs configurable for k8s Executor

• Please check this box if this contribution uses AI-generated content (including content generated by GitLab Duo features) as outlined in the GitLab DCO & CLA

What does this MR do?

Adds two new config options specifically for the Kubernetes Executor, logs_base_dir and scripts_base_dir.

These allow a user to specify a base path to prepend to the default generated paths for logs and build script storage.

Why was this MR needed?

In addition to improving the general flexibility of the runner, I am trying to use the newer User Namespaces functionality with the Kubernetes executor to improve the overall security posture of job pods, cutting their overall permissions on the host while still allowing things like package managers, etc to function correctly.

Unfortunately, when using User Namespaces, the root user of a container does not have sufficient permissions to create directories or mount volumes at the root of the container's filesystem.

This means the default paths of /scripts-$CI_PROJECT_ID-$CI_JOB_ID and /logs-$CI_PROJECT_ID-$CI_JOB_ID do not work, and result in CreateContainerConfigErrors when trying to create pods to handle a job. (This same issue also applies to builds_dir, but this is already configurable).

Using the added config options logs_base_dir and scripts_base_dir, a simple runner config can be created which works around this problem by mounting the emptyDir volume for both directories under a pre-existing path. For example:

[[runners]]
  builds_dir = "/tmp/builds" # As this issue also applies to builds_dir
[runners.kubernetes]
  logs_base_dir = "/tmp"
  scripts_base_dir = "/tmp"

Personally, I use this in conjunction with pod_spec to allow for running containers as the root user (or any user really) while allowing the cluster to map said user to a different, unprivileged UID on the host system. A sample of the config I use:

[[runners]]
  environment = ["FF_USE_ADVANCED_POD_SPEC_CONFIGURATION=true"]
  builds_dir = "/tmp/builds"
[runners.kubernetes]
  namespace = "{{.Release.Namespace}}"
  logs_base_dir = "/tmp"
  scripts_base_dir = "/tmp"
  privileged = false
  allowPrivilegeEscalation = false
[[runners.kubernetes.pod_spec]]
  name = "hostUsers"
  patch = '''
    [{"op": "add", "path": "/hostUsers", "value": false}]
  '''
  patch_type = "json"

What's the best way to test this MR?

• Build new runner and helper images, deploy using one of the above sample configs.
• Create a CI job tagged for the new runner
• Use kubectl etc. to fetch the YAML for the job pod
• Verify that the mountPath for the scripts and logs volumes match the format <logs/scripts_base_dir>/<logs/scripts>-$CI_PROJECT_ID-$CI_JOB_ID

I can also provide the above information/screenshots from my cluster.

What are the relevant issue numbers?

While looking for existing issues, I found #37760 (closed) opened recently. They may have a slightly different use case, but I am curious if this implementation still fits. Otherwise I can either update this implementation or open a separate issue.

close #37760 (closed)