GitLab Runner Helm Chart - Separate nodeSelector and tolerations for job pods
Description
In short, I think that the GitLab runner itself, and the corresponding pods that are spun up for pipeline jobs, should have separate nodeSelector
s and toleration
s.
Proposal
The current Helm chart has a simple block for the nodeSelector
in the deployment.yaml
file:
{{- if .Values.nodeSelector }}
nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
{{- end }}
And also for tolerations
:
{{- if .Values.tolerations }}
tolerations:
{{ toYaml .Values.tolerations | indent 8 }}
{{- end }}
These control placement of what I call the "master" GitLab runner instance; i.e., the pod that listens for new jobs in GitLab, and spins up the individual pipeline jobs in brand new pods. If it is possible, there should also be nodeSelector
and tolerations
blocks available for the corresponding pipeline job pods.
The reason is simple. Say I have two node groups in Kubennetes: one for generic workloads, and one specifically for GitLab runners. Ideally the GitLab runner group should have 0
instances during non-work hours (if that's desired, usually it is), but the master should always be listening for new jobs, and should scale up as needed during the day. However, as it currently stands, the GitLab runner group I have mentionted can never go to 0
because the master GitLab runner and job pods share the same nodeSelector
and toleration
blocks. This means that the master would potentially be always running on a more expensive node type, even though it's non-working hours.
What I propose is that these are separated so you can have the best of both worlds:
- Master GitLab runner pods constantly listening for jobs on regular nodes (or whatever nodes you want with the master
nodeSelector
) - Job pods spun up on nodes using the job
nodeSelector
, with the potential that the node group can scale down to0
when not needed.
Links to related issues and merge requests / references
I think this relates to the following issues:
Also, it looks like the code in the GitLab runner itself is already present (see here: #2200 (closed)). However, I'm not sure how to inject that with the Helm chart.