Concurrent execution of jobs not working when runner configuration set to have pvc attached to it
Concurrent execution of jobs not working when runner configuration set to have pvc attached to it because a single pvc cannot be attached to multiple pods at the same time and each job spins up a new pod.
https://docs.gitlab.com/runner/executors/kubernetes.html#configure-volume-types
Sample case:
Sample configuration part of gitlab runners helm chart values file.
concurrent = 4
[[runners]]
executor = "kubernetes"
[runners.kubernetes]
[[runners.kubernetes.volumes.pvc]]
name = "pvc-1"
mount_path = "/path/to/mount/point1"
In this case we have the concurrency set to 4 meaning Max 4 jobs can be executed by the runner at the same time. And consider a situation where we have 4 jobs in a single stage in the gitlab ci pipeline. So the runner will spin up 4 pods each one executing the 4 jobs respectively.
Sample gitlab ci
stages:
- build
- test
build_job1:
stage: build
script:
- sleep 60
build_job2:
stage: build
script:
- sleep 60
build_job3:
stage: build
script:
- sleep 60
build_job4:
stage: build
script:
- sleep 60
So in this case the runner will spin up 4 pod for executing the 4 jobs of which 1 pod will be in Running state and others in Init:0/1 state
When we describe the init state pods we will get the message that
Warning FailedAttachVolume
Multi-attach error for volume pvc-hzhdb-zzz.
Volume is already used by pods (pod for job1)
So in this case Concurrent execution becomes of no use.
Is there any way let say if there is concurrency set to 4 and then we will be able to add 4 pvc's to the configuration part of the gitlab runner helm chart values file such that the runner will pic up one of the 4 pvc's and assign it to the pod spinned up for the jobs in a random or rotational manner respectively, so that we can actually have Concurrent execution of the jobs?