Skip to content

Draft: Add remote cache support

Jürg Billeter requested to merge juerg/remote-cache into master

This replaces the pull-artifact-files config option with a remote-cache config option to properly support remote execution without downloading blobs that are not needed locally.

With pull-artifact-files disabled, artifact file blobs are completely ignored in cache checks. While this allows remote execution builds without downloading build outputs, it can result in build failures when blobs expire on the server side. It also does not download missing blobs on demand. I.e. bst artifact checkout and other commands don't work with pull-artifact-files disabled.

This branch replaces the pull-artifact-files support with a new remote-cache config option. When that's configured, all blobs in that remote CAS server are considered cached and only downloaded to the local system when needed (e.g. bst artifact checkout). This works for build outputs with remote execution as well as sources in remote asset servers. Unlike pull-artifact-files this still checks whether the artifact file blobs are available (using FindMissingBlobs without actually downloading them).

  • Add/update documentation for remote-cache configuration
  • Improve test infrastructure to allow proper testing of remote cache (RE CAS server isolated per test case)

This depends on !2090.

Merge request reports