Add support for standalone configuration / multiple [[ runners ]] blocks

Goal

This MR aims to add support for importing a standalone config.toml file, for use cases where using --config-template is not required/possible (runner already registered or multiple [[ runners ]] sections).

Background

The use case at my company that triggered the work on this MR is a migration from GitLab Runner "orchestrator" instances hosted on AWS EC2 to the Helm Chart. We had multiple runners declared on a single instance where each targeted a specific part of our CI/CD workflow for better isolation using tags. Jobs were executed on docker+machine executors. Runner registration is historically managed in another part of the process.

We would like to be able to keep this model without needing separate Helm deployments for each runner typology. This, for two main reasons :

1. maintainability and minimal disruption : keep a known model, easier for everyone to understand
2. resources : one GitLab Runner per actual runner means redundant reserved resources. Especially as we run our workloads on EKS, we have a limited amount of network interfaces with security groups allowed on each node.

Our use case

• We have a centralized Terraform stack which is used to register our runners on our GitLab instance.
• Using the output of the created resources and other variables (for our own configuration matters, such as caching and default images), we generate our complete TOML file.
• This generated file is then injected as a value in the Helm

How can this MR fulfill this

As of now, it is not possible to have multiple [[ runners ]] blocks in the config.toml file generated by the Helm Chart. The runners.config key is injected as a template, thus actually allows only one runner to be registered at runtime.

The chart adds a tomlConfig key in the values.yaml file, allowing the end user to put a TOML file (as a heredoc) containing the complete and final runners configuration (excluding global configuration, see end of paragraph), including runner tokens (removing the need of giving a registration token in the Helm values).

If the tomlConfig key is defined with a value, then the register-the-runner block is ignored and not executed, as the functionality implies that all runners in the file are already registered.

Global configuration values (concurrent, logLevel) are still used and the tomlConfig key is appended to them.

This is an exemple of what tomlConfig looks like :

[[ runners ]]
  name = "cd-kube-linux-x86"
  url = "http://gitlab.example"
  token = "REDACTED"
  limit = 90
  output_limit = 33554432
  executor = "kubernetes"
  environment = [""]
  [ runners.kubernetes ]
    image = "alpine:latest"
    service_account = "gitlab-runner-cd"
    cpu_request = "1000m"
    cpu_limit = "1000m"
    memory_request = "2Gi"
    memory_limit = "2Gi"
    namespace = "gitlab-runner"
    [ runners.kubernetes.node_selector ]
      "node-family"="gitlabrunner-cd"
    [ runners.kubernetes.pod_labels ]
      "app"="gitlab-runner"
      "type"="cd"

[[ runners ]]
  name = "ci-kube-linux-x86"
  url = "http://gitlab.example"
  token = "REDACTED"
  limit = 30
  output_limit = 33554432
  executor = "kubernetes"
  [ runners.kubernetes ]
    image = "alpine:latest"
    service_account = "gitlab-runner-ci"
    cpu_request = "1800m"
    cpu_limit = "3000m"
    memory_request = "7Gi"
    memory_limit = "16Gi"
    namespace = "gitlab-runner"

      
    [ runners.kubernetes.node_selector ]
      "node-family"="gitlabrunner-ci"
    [ runners.kubernetes.pod_labels ]
      "app"="gitlab-runner"
      "type"="ci"

And this is what the final file at /home/gitlab-runner/.gitlab-runner/config.toml looks like :

concurrent = 100
check_interval = 10
log_level = "INFO"
log_format = "runner"
listen_address = ':9090'
[session_server]
  session_timeout = 1800
  listen_address = "0.0.0.0:8093"
  advertise_address = "10.20.102.206:8093"

[[ runners ]]
  name = "cd-kube-linux-x86"
  url = "http://gitlab.example"
  token = "REDACTED"
  limit = 90
  output_limit = 33554432
  executor = "kubernetes"
  environment = [""]
  [ runners.kubernetes ]
    image = "alpine:latest"
    service_account = "gitlab-runner-cd"
    cpu_request = "1000m"
    cpu_limit = "1000m"
    memory_request = "2Gi"
    memory_limit = "2Gi"
    namespace = "gitlab-runner"
    [ runners.kubernetes.node_selector ]
      "node-family"="gitlabrunner-cd"
    [ runners.kubernetes.pod_labels ]
      "app"="gitlab-runner"
      "type"="cd"

[[ runners ]]
  name = "ci-kube-linux-x86"
  url = "http://gitlab.example"
  token = "REDACTED"
  limit = 30
  output_limit = 33554432
  executor = "kubernetes"
  [ runners.kubernetes ]
    image = "alpine:latest"
    service_account = "gitlab-runner-ci"
    cpu_request = "1800m"
    cpu_limit = "3000m"
    memory_request = "7Gi"
    memory_limit = "16Gi"
    namespace = "gitlab-runner"

      
    [ runners.kubernetes.node_selector ]
      "node-family"="gitlabrunner-ci"
    [ runners.kubernetes.pod_labels ]
      "app"="gitlab-runner"
      "type"="ci"