Fix issue with _pretty_size with large numbers of bytes

See merge request !799

Replaces the custom one which was erroneously added during development.
Fixes #746.

_sandboxremote.py: Use the standard SandboxError

Closes #746

See merge request !931

Update our instructions/examples regarding how we declare remote caches

See merge request !932

There are arguments for and against using onerror - see issue #735
for details.

Use _force_rmtree instead of custom code.

Closes #735

See merge request !927

Valentin David authored Nov 06, 2018 and richardmaw-codethink committed Nov 07, 2018

Root directory was marked as a non-artifact mount, so not using
SafeHardLink.  However integration commands executed with write access
to the root directory.

Fixes #749

Fix cache corruption by scripts when layout and integration commands are used

Closes #749

See merge request !934

Valentin David authored Nov 01, 2018 and Valentin David committed Nov 08, 2018

Fixes #645.

Fix infinite recursion in default strip debug command

Closes #645

See merge request !918

Benjamin Schubert authored Nov 07, 2018 and Benjamin Schubert committed Nov 08, 2018

Remove the bwraps checks from _site.py and put them in platform.linux
which is the only place where they are run.

This allows the removal of a double level of caching, making
reasoning about tests easier

Benjamin Schubert authored Nov 01, 2018 and Benjamin Schubert committed Nov 08, 2018

This removes the `_instance` on the platform object that we use
for caching and not recreating the object everytime at the start
of every test.

This is to ensure our tests share the least amount of state.

The performance penalty is from 5 to 10% accross the whole test suite.
The readings were done 5 times for each before and after the change
and on the same computer.

Benjamin Schubert authored Nov 01, 2018 and Benjamin Schubert committed Nov 08, 2018

This adds a `reason` to the SandboxEror thrown in sandboxdummy
to be able to understand where the error comes from

test behavior when there is no sandbox on Linux

Closes #696 and #736

See merge request !921

The issue was introduced by 6ccfab0b.

Fix bug with root mounted as non-artifact in script plugin.

Closes #756 and #749

See merge request !941

Angelos Evripiotis authored Nov 07, 2018 and Angelos Evripiotis committed Nov 08, 2018

As someone coming from GitHub to GitLab, I was pleasantly surprised by
the 'filter by WIP status' option.

To make sure we get the most out of it, add a guideline to keep the
filter clean, to reduce reviewer burden.

Angelos Evripiotis authored Nov 07, 2018 and Angelos Evripiotis committed Nov 08, 2018

I was recently surprised that we don't prefer addressing review comments
in 'fixup!' commits.

Coming from GitHub, I've found that fixup commits make it easier for
reviewers to see what has changed since their last review. The idea is
to use '--autosquash' before landing to clean up the history again.

It's a pleasant surprise that it's easy to keep track of what changed in
merge-requests between pushes, so we can always keep the history clean.

Document this, so folks like me can see the light sooner.

Angelos Evripiotis authored Nov 07, 2018 and Angelos Evripiotis committed Nov 08, 2018

Explain why it's useful to mention the decisions made for a change and
provide links for background, e.g. issue numbers.

My general experience of git histories is that folks don't do enough
explaining, so it's great we have a contributing section for it.

I've found it's easier for folks to stick to rules when they know the
practical reasons for them, so provide some.

Add more to GitLab-relevant parts of contributing

See merge request !935

James Ennis authored Nov 07, 2018 and Javier Jardón committed Nov 08, 2018

This patch partially resolves #700

James Ennis authored Nov 08, 2018 and Javier Jardón committed Nov 08, 2018

Due to the changed Exception message, this patch also changes
the test_parse_size_over_1024T test in misc.py

Add local cache expiry documentation and fix misleading error message when specifying a percentage cache quota

Closes #700

See merge request !939

Previously, debug messages from `Plugin.__del__()` went to stdout
which was a problem for the test suite in extremely rare and hard to
predict circumstances.  This corrects that by sending the message to
stderr where all the rest of the messages tend to go.

Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>

plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`

See merge request !944

Fixes issue #574.

This will now return paths in the same order as list_relative_paths.

Direct CAS-to-CAS import

Closes #574

See merge request !911

As per PEP 0479 (https://www.python.org/dev/peps/pep-0479/),
StopIteration thrown in context managers are not valid starting from
Python 3.7.

source.py: don't let StopIteration propagate to silence() contextmanager

See merge request !945

Phillip Smyth authored Nov 07, 2018 and Jürg Billeter committed Nov 12, 2018

_project.py: Added validate_nodes() helper function to prevent duplicate lists

element-path was being used before node validation resulting in uncaught errors

Element path not validated before use

See merge request !937

Valentin David authored Nov 12, 2018 and Valentin David committed Nov 14, 2018

Fixes #755

Run tests on aarch64

Closes #369 and #755

See merge request !948

It's inconvenient to have to create a Linux platform to parse the bwrap version
and we want to get the version in a consistent manner.

If `bwrap` fails to set up the sandbox and start the payload command
it won't write an exit-code in --json-status-fd,
so we can report if it was a sandboxing failure if we don't get exit-code status
and a payload command failure if we do and it's non-zero.

Closes #286

Distinguish between bubblewrap sandboxing failure and command failure

Closes #286

See merge request !868

of which is set to False via the addition of pullbuildtree to
userconfig.yaml

_frontend/app.py & cli.py: Add --pull-build-trees as a bst main
option, which when passed will override the default or user defined
context for pull_buildtrees

tests/completions/completions.py: Update for the added flag

The addition of cached build trees being included in element
artifacts has led to mostly redundant download overheads when
pulling from a remote artifact server. As such the default behaviour
of pull shouldn't fetch the build tree object if available.

pullqueue.py: add extensible attributes to determine required/
excluded directories for element pull jobs. scheduler context
is used to determine pull_build_trees context.

element.py: extend relevant pull logic for specified subdir
consideration and ensure push logic does not lead to partial
artifact pushes. Change assumption that buildtree can be
extracted if artifact is cached.

_artifactcache/: artifactcache.py & cascache.py inclusion of
helper functions for subdir artifact checking & fetching, fetch
logic extended to only pull required artifact directories.
extract & checkout updated to handle full/partial operation.

tests/: addition of integration test pullbuildtrees.py,
buildtrees.py adapted cli options, testutils/artifactshare.py
has_artifact changed to return artifact digest on true condition.