Support multiple runners sections in the configuration template
Overview
Customers using the Runner on K8s need to be able to easily install multiple Runners on the K8S coordinator pod to efficiently use the cluster resources.
Context
"The gitlab-runner is able to manage multiple [[runners]]
at once. Having a single instance of gitlab-runner for many [[runners]]
will help to optimize the usage of the K8S cluster: a single gitlab-runner knows exactly how many jobs are currently running, even across multiple registrations.
Here is the scenario: I have a single K8S cluster and I wish to offer a gitlab-runner for 3 groups of projects.
Manually, I would run a single gitlab-runner, with a concurrency=10 (for any reason, I know that 10 jobs max is good for the cluster). And then, the gitlab-runner will pick the jobs from the 3 groups, up to 10 jobs in parallel. It could be 6 of the first group, 3 of the second and one of the third.
Using Gitlab Runner Operator or the current Helm Chart, I cannot optimize like this. I have to choose how to split the "10 jobs" limit.
If I instantiate 3 gitlab-runner with concurrency=10 each, it could be possible that 30 jobs are picked by the runners, and the cluster will not be able to support them.
If I instantiate 3 gitlab-runner with concurrency=3 each, if only one group has more than 3 jobs to process at a given time, they will be queued while the cluster has remaining slots.
PS : For such design, think of the ingress-nginx controller: a controller with a single nginx instance, managing multiple Kind=Ingress in order to generate a single config file. I think it is clearly possible to do the same think here."
Proposal
Today, either with Helm or Operator we only support installing one Runner coordinator per a K8S coordinator pod. With this feature, then users can define multiple runner sections in the configuration template