Distributed Cache Not Working with S3 Compatible Ceph Object Gateway
Summary
I'm trying to setup Distributed Cache in my Gitlab Runner in order to speed up the execution of Docker Build. For this I plan to utilise Ceph Object Gateway which is a S3 compatible Object Store like MiniO. However even after updating information related to Ceph, Gitlab Runner doesn't seem to be utilising it at all. I'm not seeing anything in logs wherein it even tries to utilise Ceph to store anything.
Steps to reproduce
Test Gitlab Runner distributed cache on Ceph Object Gateway (Version - Octopus 15.2.8)
Gitlab Runner Helm Chart values.yaml
gitlabUrl: https://10.10.0.0/
runnerRegistrationToken: "xxxxx"
unregisterRunners: true
terminationGracePeriodSeconds: 3600
certsSecretName: 10.10.0.0-cert
concurrent: 10
checkInterval: 30
rbac:
create: true
clusterWideAccess: false
podSecurityPolicy:
enabled: false
resourceNames:
- gitlab-runner
metrics:
enabled: true
runners:
config: |
[[runners]]
name = "Kubernetes Runner"
executor = "kubernetes"
[runners.kubernetes]
image = "alpine:latest"
privileged = true
cpu_request = "6"
cpu_limit = "10"
memory_request = "6Gi"
memory_limit = "10Gi"
ephemeral_storage_request = "10Gi"
ephemeral_storage_limit = "20Gi"
helper_cpu_request = "6"
helper_cpu_limit = "10"
helper_memory_request = "6Gi"
helper_memory_limit = "10Gi"
helper_ephemeral_storage_request = "10Gi"
helper_ephemeral_storage_limit = "20Gi"
service_cpu_request = "6"
service_cpu_limit = "10"
service_memory_request = "6Gi"
service_memory_limit = "10Gi"
service_ephemeral_storage_request = "10Gi"
service_ephemeral_storage_limit = "20Gi"
poll_interval = 5
poll_timeout = 3600
dns_policy = "default"
[dns_config]
nameservers = [
"10.10.0.1",
]
[runners.cache]
Type = "s3"
Path = "/cache/"
Shared = true
[runners.cache.s3]
ServerAddress = "http://10.10.0.2:8080"
AccessKey = "xxxxx"
SecretKey = "xxxxx"
BucketName = "gitlab-runner"
BucketLocation = ""
Insecure = true
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
tags: "telco-cdl-gitlab-runner"
name: "telco-cdl-gitlab-runner"
securityContext:
fsGroup: 65533
runAsUser: 100
Environment description
I'm using Shared Runner at Group level, running using Kubernetes Executor on our K8s Cluster (On-Premise)
Used GitLab Runner version
Running with gitlab-runner 13.8.0
Preparing the "kubernetes" executor
Using Kubernetes namespace: gitlab-runner
Edited by Prateek Dubey