Draft: Add remote cache support
This replaces the pull-artifact-files
config option with a remote-cache
config option to properly support remote execution without downloading blobs that are not needed locally.
With pull-artifact-files
disabled, artifact file blobs are completely ignored in cache checks. While this allows remote execution builds without downloading build outputs, it can result in build failures when blobs expire on the server side. It also does not download missing blobs on demand. I.e. bst artifact checkout
and other commands don't work with pull-artifact-files
disabled.
This branch replaces the pull-artifact-files
support with a new remote-cache
config option. When that's configured, all blobs in that remote CAS server are considered cached and only downloaded to the local system when needed (e.g. bst artifact checkout
). This works for build outputs with remote execution as well as sources in remote asset servers. Unlike pull-artifact-files
this still checks whether the artifact file blobs are available (using FindMissingBlobs
without actually downloading them).
-
Add/update documentation for remote-cache
configuration -
Improve test infrastructure to allow proper testing of remote cache (RE CAS server isolated per test case)
This depends on !2090.