Abstraction of storage used by sandbox

As part of the work to include content-addressable storage as a backend for BuildStream (see also #387 (closed)) we need to abstract the use of the sandbox's storage so BuildStream and plugins can use storage without being coupled to a specific implementation.

Create a virtual directory class which acts as an interface to the storage used by sandboxes and alter all existing code to use it instead of handling files and directories.

Also add a plugin configuration variable called BST_VIRTUAL_DIRECTORY which indicates whether a plugin uses the new interfact, and which we can use to dynamically choose a backend for a particular plugin.

!445 (merged) will implement this.

marked this issue as related to #387 (closed)

mentioned in issue #417 (closed)

added Verify label

!445 (merged) is ready for first review.

unassigned @jmacarthur

added Doing and removed Verify labels

assigned to @jmacarthur

Needs extra work to support tarball checkout which was recently merged. Also some review comments raised by @juergbi to address.

added Important label

!445 (merged) has been reviewed by @danielsilverstone-ct and I have updated the MR in response. Daniel has indicated that he would like to re-review the changes before approving.

Merged.

added Verify and removed Doing labels

closed

removed Verify label