Fixes: e29aea36 ("Basic options for shell --build to use buildtrees")

_frontend/app.py: Use buildtree for interactive shell on build failure

See merge request !1039

  * utils.py:_kill_process_tree(): Ignore NoSuchProcess errors

    These are caused because we issue SIGTERM, and if the process
    has not exited after a timeout, we kill it.

  * _scheduler/jobs/job.py: Stop handling NoSuchProcess errors here
    redundantly, they are already ignored.

It seems that we were ignoring it after sleeping when terminating
tasks from the scheduler... but we were not ignoring it when performing
the same pattern in the `Plugin.call()` -> `utils._call()` path, so
we were still getting these exceptions at termination time from host
tool processes launched by source plugins.

Fix stack traces discovered with ^C forceful termination.

See merge request !1043

This is not used anywhere outside of the Scheduler, currently
only the Scheduler itself is allowed to queue a job at this level.

If the highlevel business logic for automatic queueing of auxiliary
jobs moves to another location, we can make this public again.

When queuing the special cache management related cleanup and
cache size jobs, we now treat these jobs as special and do the
following:

  * Avoid queueing a cleanup/cache_size job if one is already queued

    We just drop redundantly queued jobs here.

  * Ensure that jobs of this type only run one at a time

    This could have been done with the Resources mechanics,
    however as these special jobs have the same properties and
    are basically owned by the Scheduler, it seemed more straight
    forward to handle the behaviors of these special jobs together.

This fixes issue #753

This changes the deepest callback from when a Job completes to
propagate a JobStatus value instead of a simple boolean, and updates
all of the effected code paths which used to receive a boolean
to now handle the JobStatus values.

This further improves the situation for issue #753, as now we avoid
queueing cache size jobs for pull jobs which are skipped.

This is redundant now that we report it through the JobStatus.

Only queue one cache size job

Closes #753

See merge request !1040

Since we added batch commands, the batch commands print the
text of the commands directly in the message text, but this is wrong.

The detail string is the appropriate place for text of unknown lengths
(the user can actually configure how many max lines of commands they
want to see in their log), the message text itself should be controlled
and brief enough to avoid text wrapping.

sandbox/sandbox.py: Fix regression of command logging

See merge request !1044

Phil Dawson authored Jan 07, 2019 and Jürg Billeter committed Jan 08, 2019

This command has been replacved by the bst source checkout command

element.py: remove documentation reference to source bundle command

See merge request !1041

When code is faulty, in a plugin or in the core, we should
get a stack trace and a BUG message.

sandbox/sandbox.py: Use assertions for programming errors instead of BstErrors.

See merge request !1046

Valentin David authored Dec 20, 2018 and Valentin David committed Jan 09, 2019

Fixes #631.

Valentin David authored Dec 20, 2018 and Valentin David committed Jan 09, 2019

Fixes #780.

Remote execution configuration: HTTPS and user configuration

Closes #780 and #631

See merge request !1030

ScriptElement does not use Sandbox.get_directory. It works using it
with remote execution.

Fixes #850

buildstream/plugins/elements/script.py: Mark script as BST_VIRTUAL_DIRECTORY

Closes #850

See merge request !1047

It was 'instance-name' in the documentation.

Rename 'instance_name' option to 'instance-name'

See merge request !1048

This patch namespaces the test temp directory and the output
coverage report file with the name of the environment under test,
such that separately run tests do not access the same files.

When running tests without tox, directly through setup.py,
then the tmp directory will still be `./tmp`.

  * .gitignore: Added new .coverage-reports/ directory

  * .gitlab-ci.yml: Rely on tox to combine and report coverage, only
    tell tox about the COVERAGE_PREFIX so that results can be namespaced
    by CI job name.

    This change also publishes the sources and final combined `.coverage`
    file in an output gitlab artifact for inspection, and lists some missing
    dependencies to the `coverage` job.

  * tox.ini: Add comments and refactor main [testenv] section so that
    other environments dont inherit too much unrelated cruft.

    Generate the coverate reports in the respective {envtmpdir} so that
    all per-process coverage files are prefixed with a full path, ensuring
    that concurrent runs don't mix reports and addressing concerns
    raised in #844.

    Also implemented new `tox -e coverage` environment to combine
    any found coverage and print a report.

  * .coveragerc: Omit .tox/ directory from coverage stats

Fixes issue #844

Also point out that it is possible to run test environments in
parallel using the `detox` tool.

Allow using detox, and fix/refactor collection of coverage reports

Closes #844

See merge request !1051

  * Omit versioneer's _version.py

  * Omit our __main__.py which is used only internally for generating
    documentation

.coveragerc: Omit some things which are irrelevant to cover.

See merge request !1052

Accept common architecture aliases for arch options instead of only
accepting the canonicalized, OS-independent architecture name. This
restores compatibility with existing projects and may simplify option
handling for projects that only target a single OS (and thus, do not
need OS-independent architecture names).

Accept common architecture aliases for the sandbox config for
consistency with arch options.

Accept architecture aliases

See merge request !1034

contributing: fix 'oprtation' and some other typos

See merge request !1055

The algorithm adds elements to a dictionary and then sorts the dictionary
by the discovered depths while recursing - using an OrderedDict is enough
to ensure a stable order.

This fixes `bst show --deps plan ...` reporting different results on
each invocation.

This fixes an aspect of #824

Added a function to report the list of elements which appeared in
a given queue in the order of their first appearance.

For each sample project and expected result order, this test ensures that:

  * bst show --deps plan: Always shows the expected build order.

  * bst source fetch: Always executes in the expected build order.

  * bst build: Always builds in the expected order (disabled).

The build part of the test is currently disabled as the scheduler
seems to be messing with the order even in a `--builders 1` scenario.

Make build plan element list stable

See merge request !1058

Angelos Evripiotis authored Jan 11, 2019 and Angelos Evripiotis committed Jan 11, 2019

Bump the version of pytest-cov, so that we pick up the fix for
the '--no-cov' option breaking in pytest 4.0:

    https://github.com/pytest-dev/pytest-cov/pull/230

This was the error you would get prior to this version:

    _pytest.warning_types.RemovedInPytest4Warning: config.warn has been
    deprecated, use warnings.warn instead

Although it says 'deprecated', it did result in
termination with stack trace.

Angelos Evripiotis authored Jan 11, 2019 and Angelos Evripiotis committed Jan 11, 2019

To help new folks get to grips with pytest, add some tips on other
frequently used features.

pytest-cov==2.6.1, contributing: '--last-failed' and '--no-cov' tips

See merge request !1059

Minor correction, looks like we're not observing this queue
otherwise we'd be seeing crashes.

_scheduler/queues/queue.py: Put elements in the skip list, not jobs

See merge request !1063

.gitlab-ci.yml: Use latest freedesktop-sdk 18.08.25

Closes #858

See merge request !1064

During recent reorganizing of `.gitlab-ci.yml` to work with `tox`, seems
like we missed to update the image used by the overnight aarch tests,
meaning that they currently fail due to `tox` being missing from them.

While these tests will be skipped on MRs usually, here is an example of
what the tests look like if they are actually run with this change:
https://gitlab.com/BuildStream/buildstream/-/jobs/145449561.

Fixes #859.

Chandan/fix overnight aarch

Closes #859

See merge request !1065

Jürg Billeter authored Jan 10, 2019 and Tristan Van Berkom committed Jan 14, 2019

The path of the config file generated by testutils for completion tests
is passed as regular argument, not via COMP_WORDS. Use that config file
in complete_artifact() to ensure the test uses the right artifact
directory.

James Ennis authored Jan 09, 2019 and Tristan Van Berkom committed Jan 14, 2019

Since the artifact subcommand group has been created, we have been able
to tab complete both element names and artifact refs as arguments to
the artifact subcommands (e.g. bst artifact log), this commit adds a
test for this.

completions.py: Add a test for our artifact ref autocompletions

See merge request !1054

This will help us check if BuildStream is working with the latest
version of dependencies, as per our constraints. This job is allowed to
fail but its failure should signal that we need to add stricter
constraints in some of our `.in` requirements files.