Skip to content

Don't pull artifact buildtrees by default

Tom Pollard requested to merge tpollard/494 into master

Description

Don't pull artifact buildtrees by default. As buildtrees are now added at caching to element artifacts their size in some cases has significantly increased. When relying on a remote artifact cache to pull this leads to longer fetch times which can be severely worsened depending on the users bandwith. The buildtrees in most cases (especially of depedencies) are not needed when building targets, however there needs to be support for adding buildtrees to artifacts for certain actions. This is discussed further in #494

- Set default pull to not include buildtree artifact dir
- configurable subdir and excluded_subdirs
- Add 'pull-buildtrees' global cache group option for user conf and userconfing.yaml default
- Add --pull-buildtrees flag to bst main options, override user conf default
- Add helper function _fetch_subdir to cascache, to fetch buildtree
  or any other subdir digest, also adds a function to check if a cached subdir exists
- Make element._pull_pending not assume no need to process pull if
  artifact is cached if buildtrees are set to be pulled. _skip_push updated to not allow
  partial pushes
- Ensure cascache.py doesn't try to checkout/extract a dangling ref, whilst 
  also being able to checkout and extract just the buildtree if the rest of the artifact has 
  already been extracted from the cache.
- Required test and changes

This merge request, when approved, will partly close issue/bug: #494

Edited by Tom Pollard

Merge request reports