Use buildbox-casd for CAS access
This implements the changes described in #1042.
Task list
-
Fix jobpickler test failure after rebase -
Fix test_{missing,old}_brwap_has_nice_error_message -
Fix test_dummy_sandbox_fallback -
Fix test_host_tools_errors_are_not_cached -
Fix new virtual directory tests after rebase -
Get BuildBox MRs merged -
CI -
Fix tests with buildbox-fuse sandboxing backend -
Fix flaky test_never_delete_required_track -
Fix test_junction_build_remote (!1527 (merged)) -
buildstream-docker-images!125 (merged) Add buildbox-casd to testsuite docker images -
Use new testsuite docker images -
Add buildbox-casd to WSL CI
-
-
Final branch rebase (ensure testsuite passes for every commit) -
Update and test BuildStream install instructions to include buildbox-casd: website!123 (merged) -
Benchmark
Post-merge fixes
I expect it to be acceptable to merge this branch without the following fixes, however, they should be fixed soon.
-
Fix test_invalid_cache_quota -
Restore 'Cache Usage' display in frontend
Additional optimizations and other improvements are planned to land after merging this branch. See #1042 for a tentative task list.
Note to reviewers
As the branch consists of a relatively large number of commits, here a brief overview of the commit structure:
- Run internal CAS tests in subprocesses as local CAS cache writes are no longer allowed from the main process with buildbox-casd (gRPC)
- Remove existing CAS expiry support as cache expiry will be managed by buildbox-casd. This marks corresponding tests as xfail.
- Add and use CASCache.cleanup() as preparation to always properly terminate buildbox-casd.
- Minimal buildbox-casd dependency: Update CI images, start/stop buildbox-casd process and set up channel
- Use buildbox-casd for local cache (write) operations.
- Enable local cache expiry via buildbox-casd, remove xfail mark from expiry tests.
- Use buildbox-casd for remote CAS operations.
- Enable cache expiry via buildbox-casd in bst-artifact-server.
Edited by Jürg Billeter