Old versions of git lack --force and --tags,
but the same effect can be had by specifying refspecs.

The test assumes that a directory with write permission removed isn't writable,
this isn't the case if the process running the tests has CAP_DAC_OVERRIDE
which is common when running as root.

test_track_invalid_submodule depends on being able to remove a submodule
by `git rm $submoduledir`, but old versions of git don't update .gitmodules
so BuildStream still thinks there's a submodule present.

For expediency the test is skipped rather than changed to manually remove
the entry from .gitmodules if git hasn't done it,
since in the common case git is new enough to do that itself.

test_git_describe expects --first-parent to find another tag,
but `bst track` will gracefully degrade if the option doesn't work
so a different history will be retained with old versions of git.

It's of marginal benefit to add additional cruft
to test for different output on old versions of git that won't persist forever.

Benjamin Schubert authored Dec 20, 2018 and richardmaw-codethink committed Jan 21, 2019

Centos is apparently different enough from fedora when running tests.

Fix CentOS

Closes #833

See merge request !1085

The link explaining python's requirements files was formatted
incorrectly (missing underscore at the end). This resulted in the text
being rendered literally, as opposed to being converted into a
hyperlink.

CONTRIBUTING.rst: Fix formatting of link to pip docs

See merge request !1094

This commit also ensures that if we try to use the 'old' commands,
BuildStream will fail and instruct the user to use the new command.

'artifact checkout' has slightly different behaviour from 'checkout',
that is, either '--directory' or '--tar' are now required options.
This is a step towards allowing checkout to take multiple args.

This commit marks 'bst checkout' as a 'hidden' command. If used,
the user will be prompted to use the new 'bst artifact checkout'
command.

All tests which used 'bst checkout' have been modified to use
the new artifact sub-command.

This partially solves #822.

Due to an upstream click_man issue:
https://github.com/click-contrib/click-man/issues/10
generating the man pages with our current setup.py only generates
a man page for bst-artifact-server, our first entry point.
I then had to remove this entry point from setup.py and regenerate to
obtain man pages for the commands in cli.py

Move push/pull/checkout to the artifact subcommand group

See merge request !1045

Click 7.0 allows us to declare commands as 'hidden'.
However, sphinx-click and click-man still generate these hidden
commands in the documentation and man pages, respectively.

This is a stop gap solution until the upstream issues (tracked in
issues #879 and #881) have been addressed.

Add an obsoletion note to our deprecated commands.

See merge request !1097

This fixes a recent regression introduced in c2fc2a5e

_scheduler: Fix dont display a failure for terminated jobs

See merge request !1096

Instead of sitting mysteriously alone in internals/utils.py

This is not an error related to loading data, like a parse error
in the quota specification is, but a problem raised by the artifact
cache - this allows us to assert more specific machine readable
errors in test cases (instead of checking the string in stderr, which
this patch also fixes).

This also removes a typo from the error message in the said error.

  * tests/artifactcache/cache_size.py

    Updated test case to expect the artifact error, which consequently
    changes the test case to properly assert a machine readable error
    instead of asserting text in the stderr (which is the real, secret
    motivation behind this patch).

  * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota()

    Now expect the artifact error for the tests which check configurations
    which create caches too large to fit on the disk.

Don't require the interest to be registered, just discard any interest,
this function just sets a bit in a mask, and is not intended to maintain
a balance like the reserve() function is.

When running any session that has Queues which require Resource.CACHE,
check if our loaded estimated size exceeds the quota, and if so;
lock the Resource.CACHE resource exclusively right away and run
an exclusive initial cache size job.

This ensures we cleanup first before doing anything which might
add to the cache at startup time, if deemed needed.

This is a partial fix for issue #737

In order to test when core activities occur by parsing the stderr
in tests, we should make the messages conform more.

At the same time, this restores alignment of columns in core
messages with the element processing related messages.

Also, _scheduler/scheduler.py is updated to make it's activity names
conform to the (current) 5 character limit for the sake of alignment.

The tests/frontend/logging.py test gets it's regex updated for
the log line it checks for in stderr.

Instead of a single target, we can always provide a single target
in a list.

The artifact cache emits messages, and we want to allow that in preflight.

We can streamline this call to os.statvfs() in a few places.

This will benefit from a better UtilError being raised, and
and turns the artifact cache's local function into a one liner.

The loop which finds the first existing directory in the
given path has been removed, being meaningless due to the
call to os.makedirs() in ArtifactCache.__init__().

The local function was renamed to _get_cache_volume_size() and
no longer takes any arguments, which is more suitable for the
function as it serves as a testing override surface for
unittest.mock().

The following test cases which use the function to override
the ArtifactCache behavior have been updated to use the new
overridable function name:

   tests/artifactcache/cache_size.py
   tests/artifactcache/expiry.py

A simple object which creates a snapshot of current
usage statistics for easy reporting in the frontend.

A frontend facing API for obtaining usage statistics.

I would have put this on Stream instead, but the Context
seems to be the de facto place for looking up the artifact cache
in general so let's put it here.

This also adds some comments around the main status bar heading
rendering function.

Added some useful status messages when:

  * Calculating a new artifact cache usage size

  * Starting a cleanup

  * Finishing a cleanup

Also enhanced messaging about what was cleaned up so far when
aborting a cleanup.

Updates the known cache size in the main process while the cleanup
process is ongoing, so that the status indicators update live
while the cleanup happens.