Skip to content

Abstract directory class and filesystem-backed implementation

This is the virtual 'Directory' class and a filesystem-backed implementation of it. The intention is to create a common interface which the content-addressable storage system can also implement. See Juerg Billeter's email 'Proposal for Remote Execution' to the BuildStream mailing list for more details. This implementation has deviated from the original proposal in some ways:

  • I've tried to keep the full path to the underlying sandbox directory a secret held only by classes in the sandbox directory to stop any unexpected access to the file system. However, there are some issues with this - I realised late that what's called the 'root directory' of the sandbox, i.e. that previously returned by get_directory, is not actually the toplevel directory held by the sandbox. At the moment, the sandbox root is virtualised, but other directories are not. Some of those directories need to be exposed externally, e.g. 'logs'.

  • Because we kept the artifacts/files directory outside the virtualised root, we now need export_files to move things out of the virtual directory and into the real directory used for artifacts.

  • Once import and export were implemented, there wasn't any need for functions which move/copy/rename files internally inside the virtual directory. The only case I could think of for these would be moving artifacts into artifact/files after assemble() ran, but with the directory problems above, these are now done with export_links.

  • Directory.descend creates directories when descending (or throws an exception), which is a change from the previous behaviour where paths could be composed with os.path.join without referring to existing directories.

Edited by Jim MacArthur

Merge request reports

Loading