Skip to content

CAS-backed virtual directory implementation

Jim MacArthurrequested to merge
jmac/cas_virtual_directory into master

Description

Implement CAS-backed virtual directory

Changes proposed in this merge request:

  • Add _casbaseddirectory to the previous abstract filing system in !445 (merged)

This merge request, when approved, will close issue/bug #417 (closed).

Still to do on this request:

  • Respect the files parameter for import
  • Correctly populate FileListResult
  • Existing test suite and PyLint/PEP8.
  • Additional tests for CAS-based directories.

Note that at present, the CAS-based directory is not used by default in any tests, so the code is not tested by CI. There is a hidden environment variable called BST_CAS_DIRECTORIES, which if set, will use CAS-based directories in place of File-based ones. This should run any test which doesn't need to run shell commands in a sandbox. The general means of testing is to run the normal test suite with this environment variable on, e.g. BST_CAS_DIRECTORIES=1 ./setup.py test. Since the existing test suite exercises many aspects of the virtual directory interface, there didn't seem much benefit in duplicating tests to run on the CAS backend instead.

I've dropped the requirement to perform CAS-to-CAS import since it's a very large amount of work to implement correctly. This is partially because we don't define import order correctly in the existing BuildStream code; trying to replicate that ordering is very difficult because of nondeterminism in the existing list_relative_paths. #574 (closed) has been raised to add directory CAS-to-CAS import later.

Edited by Jim MacArthur

Merge request reports