Draft: Enable dynamic PersistentVolumeClaims in Kubernetes
What does this MR do?
Adds the capability to configure the Kubernetes executor to instantiate a VolumeClaimTemplate
whenever it creates a pod.
The template is instantiated by injecting CI variables into it. The MR also enables injecting variables
to the name of a pvc
volume attached to the pod.
By using these two changes together, you can reserve and mount volume claims for pods dynamically.
Why was this MR needed?
Currently the Kubernetes runner always clones the git repository from scratch; it has no cache.
It would be possible to configure a cache into use manually by having a single huge volume,
which is mounted in a multi-user mode and mounted to every pod. The concurrent pods could
then use different folders via CI_CONCURRENT_ID
. But multi-user volumes are slower and less
safe than single-user ones.
With the changes in this MR, it is possible to configure a volume claim template, where
the name includes CI_CONCURRENT_ID
and mount it. This allows a per-concurrent-execution
volume like in the docker executor. Under the hood these can be allocated from a
fast per-node storage class.
What's the best way to test this MR?
- Install this version of gitlab-runner on a Kubernetes cluster.
- Create or select a storage class for the build volumes. Here is an example of what worked for me
in Azure:
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: some-storage-class provisioner: disk.csi.azure.com parameters: skuname: StandardSSD_LRS allowVolumeExpansion: true # Keep the volumes around. They are used for git clones of build jobs # and we don't want to clone from scratch each time. # We may need some mechanism to ensure the system doesn't # leak these over time. reclaimPolicy: Retain # The volume is created when first used. volumeBindingMode: WaitForFirstConsumer
- Configure a pvc mounting and volume template, both with the name set to ´pvc-$CI_CONCURRENT_ID´ and configure the build directory to reside on that volume.
- Observe that repeated jobs from the same
project with
GIT_STRATEGY
set to fetch will no longer have to do the clone from scratch every time.
[[runners]]
executor = "kubernetes"
builds_dir = "/mnt/dynamic-pvc"
[runners.kubernetes]
# Creates a PersistentVolumeClaim for each concurrent job execution
[[runners.kubernetes.volume_claim_templates]]
template = """
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-$CI_CONCURRENT_ID
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
storageClassName: some-storage-class"""
# Mounts the generated PersistentVolumeClaim to a folder.
[[runners.kubernetes.volumes.pvc]]
name = "pvc-$CI_CONCURRENT_ID"
mount_path = "/mnt/dynamic-pvc"
What are the relevant issue numbers?
This MR continues on top of !4256 (merged),
which enables the injection of variables to [[runners.kubernetes.volumes.pvc]].name
.