Only download the build tree if absolutely necessary

Background

This is technical debt from landing #21 (closed) and should be addressed during the 1.3 development cycle.

Now that we're landing #21 (closed) and caching the build tree as a part of the artifact, this constitutes a major shift in how much data is uploaded, and more importantly downloaded.

The artifact share server is supposed to improve productivity such that you do not need to build things if they are already available on the share, however when given the choice between downloading many GB of object files for a WebKit build, or just building WebKit myself, I'm no longer sure I would even choose to download the whole build tree.

Task description

Ensure that downloading the build tree is only ever done either on explicit request (e.g. with a new option to bst pull and bst build), or as a result of triggering a user facing feature which requires use of a cached build tree, like when opening a workspace for a cached build (#311).

• Ensure the CAS client/server layer has support for downloading partial artifacts

• Configurable PullQueue, allow specifying whether you want to download the main artifact, the build tree, or both

• If BuildStream is going to push artifacts to different artifact server than the one it downloads from, it must download the build trees in order to upload the new artifacts; incomplete artifact uploads are not allowed.

• When implementing features such as #311 for example, we should be running the scheduler to ensure that we download the build trees as a part of the activity of opening a workspace, using the configurable PullQueue to complete the local artifacts (in this case, probably with an option for the user to decide whether they want to use the cached build tree or not).

  This task item is being addressed in #774 (closed)

Acceptance Criteria

We should not be downloading build trees unless the client needs them for some reason.