profile_start() and profile_end() calls have been incorporated into
Scheduler.run()

profile_start() and profile_end() calls have been added to
Stream.load_selection()

Add new 'scheduler' and 'load-selection' profiling topics

See merge request !1088

Instead only rely on the headroom to be enough to protect against
out of space conditions. The headroom can become configurable as
a separate step is required.

The changes to achieve this are:

  * Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().

  * ArtifactCache.full() now also reports True if the available
    space on the artifact cache volume is smaller than the headroom.

    This ensures jobs get triggered to cleanup the cache when
    reaching the end of the disk.

  * When loading the artifact quota, it is now only an error if
    the quota exceeds the overall disk space, not if it does not
    fit in the available space.

    It is still a warning if the quota does not fit in the
    available space on the artifact cache volume.

  * Updated scheduler.py and buildqueue.py for the API rename

  * tests: Updated the artifactcache/expiry.py test for its
    expectations in this regard.

    Added a new test to test an error when quota was specified to
    exceed total disk space, and adjusted the existing tests to
    expect a warning when the quota does not fit in the available
    space.

This fixes issue #733 and #869.

_artifactcache.py: Don't require the quota to be available on disk.

Closes #869 and #733

See merge request !1106

Chandan Singh authored Jan 24, 2019 and Chandan Singh committed Jan 25, 2019

Previously, one had to manually install `click-man` package, and
remember to run the correct command. Now, we can simply run `tox -e man`
to update the man pages.

Chandan Singh authored Jan 24, 2019 and Chandan Singh committed Jan 25, 2019

Notable changes:

* New `source` and `artifact` command groups
* Man pages corresponding to obsolete commands, that were marked as
  hidden in Click, have now been removed.

Fixes #881.

Chandan Singh authored Jan 24, 2019 and Chandan Singh committed Jan 25, 2019

Simplify our docs, by requesting users to run `tox -e man` to update man
pages, instead of manually installing `click-man` and running the
command manually.

Chandan Singh authored Jan 25, 2019 and Chandan Singh committed Jan 25, 2019

If the `man` directory is empty, then it won't be copied in the source
distribution, and `list_man_pages()` will throw an exception when trying
to list files in a non-existent directory. This prevents us from
installing the BuildStream package when the man pages are not there.

The most common use-case for this is when we want to re-generate the man
pages but want to install the package before re-generating them.

Generate man pages using tox & update them

Closes #880 and #881

See merge request !1107

use_artifact_config added as an optional default arg, allowing
for loading of given elements artifact remote config.

Provide bst shell --use-buildtree the ability to attempt to
acquire missing buildtrees, given respective option, user
pull-buildtree context and remote availability.

_frontend/cli.py: Refactor logic for determining --use-buildtree
option with given opportunity to attempt pulling a non-local
buildtree. Element loaded with artifact_config to allow remote
querying.

_stream.py: With given user option and element state, construct
PullQueue to fetch remote buildtree. Continue or Error without
buildtree if cannot be attained.

tests/integration/build-tree.py: Update to support new usecases