Multiple runners should be able to share cache on NFS
This functionality is requested by a customer (internal link).
Multiple runners on the same host cannot currently share their cache with each other without object storage. Similar to distributed caching's Shared
option, we should allow these runners to share their cache when using a single shared directory, such as an NFS mount.
Our docs mention the following about using cache:
From the perspective of the Runner, in order for cache to work effectively, one of the following must be true:
...
- Use multiple Runners (not in autoscale mode) of the same architecture that share a common network-mounted directory (using NFS or something similar) where the cache will be stored.
I tested the following config with each of 2 runners on the same host so they could share the /runner_cache
directory for storing their cache:
[[runners]]
name = "NAME"
url = "https://gitlab.com/"
token = "TOKEN"
executor = "docker"
cache_dir = "/runner_cache"
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "IMAGE"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/runner_cache"]
cache_dir = "/runner_cache"
shm_size = 0
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
However, it stores the cache in subdirectories unique to each runner:
/runner_cache/runner-XXXXXX-project-111111-concurrent-0/<runner-hash>/<group>/<project>/<cache-key>/cache.zip
Note that it's saved under runner-XXXXXX-project-111111-concurrent-0
- the name of this directory is unique for each runner. As a result, the cache created in the first runner is inaccessible by any job in the other one.
I discussed this with @steveazz and he found that setting distributed caching's Shared=true
in the config.toml
is ignored when object storage isn't used. Similar to the Shared
key (which only works with object storage), we should allow for runners to share their cache on a shared directory, such as a NFS mount.
I tested this on Gitlab.com and with gitlab-runner 12.3.0.