This ensures that option expressions are resolved in the project
level overrides before attempting to composite them on the instantiated
elements. Seems this is a regression from introducing the include
directive.

This fixes issue #658

This is a regression test for issue #658

Fix override options

Closes #658

See merge request !802

Since ArtifactCache.setup_remotes() can be expensive and should only
happen once, this commit will assert() if it is called a second time
on an artifact cache instance.

Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>

The initialization of remotes is done by ArtifactCache.setup_remotes()
and as such it was wrong for these tests to be calling
CASCache.initialize_remotes() a second time.

Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>

The SandboxRemote used to construct its own CASCache which was
considered dangerous.  This patch replaces that with acquisition of
the cache via the Platform singleton, hopefully eliminating issues
from having more than one artifact cache object in a single process.

Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>

sandbox/_sandboxremote.py: Acquire cache via Platform

See merge request !797

When configuring a push remote and specifying either the client-cert
or the client-key, then both must be specified. This ensures we
get an informative error instead of a stack trace and BUG.

Fixes issue #625

Test that we get the expected error when configuring a client-cert
without client-key, or the inverse.

Fix artifact config crash

Closes #625

See merge request !804

richardmaw-codethink authored Sep 04, 2018 and Tristan Van Berkom committed Sep 18, 2018

This fixes all devices being mapped to the non-existant device 0,
which prevents being able to use even safe devices like /dev/null
through the hardlinks FUSE layer.

richardmaw-codethink authored Sep 04, 2018 and Tristan Van Berkom committed Sep 18, 2018

This is needed to permit access to the device nodes added to /dev
on Linux when FUSE is used as root.

The chroot sandbox only works with all privileges,
so there's no explicit check for being root
or having the appropriate capabilities.

A check for whether it's running as root isn't needed on Linux with bubblewrap
because /dev or its devices are mounted on top of the FUSE layer,
so device nodes are accessed directly rather than through the FUSE layer.

fix chroot sandbox devices

See merge request !781

bst build returns "missing project.conf" when a project.conf is invalid
This results in an existing project with malformed yaml being dismissed
and attempts to make a new project

Added new exception

Incorrect error when malformed project.conf

Closes #642

See merge request !792

The SKIPPED message type is now used to indicate the end of a task which
was successful without having to perform the given task.

This overhauls the use of `Queue.done()` and therefore queues do not
need to provide a processed/skipped return value from `done()`. Instead
this is replaced with the action of raising a `SkipJob` exception from
within `Queue.process()`.

Adds a test to ensure that BuildStream alerts the user of a skipped push
when the remote already has the artifact cached.

This removes the timed_activity for an element _push action. This is
unnecessary as the job is already being timed elsewhere.

Pulled/Pushed messages will no longer be produced from within
element.py, instead they will be produced during CasCache push() and
pull() appropriately.

Message consistency has also been improved.

Both pulling and pushing INFO messages are now status messages.

Calls to the messaging API through `self.context.message()` have now
been switched to `element.info`.

Add SkipJob for indicating a skipped activity

See merge request !765

Tristan Van Berkom authored Sep 19, 2018 and Tristan Van Berkom committed Sep 19, 2018

This actually improves reliability of the status bar because we
now disable it completely in the case that not all of the terminal
escape sequences are supported on the given terminal.

This replaces the few functions we were using, to move the cursor
up one line, move it to the beginning of the line, and to clear a
line, with low level functions provided by the curses module in
the standard library.

This change makes it easier for downstream distro package maintainers
to package BuildStream, particularly on Fedora.

Asides from changing _frontend/status.py, this commit includes the
following changes:

  * _frontend/app.py: Use python isatty() function to determine if
                      we are connected to a tty, instead of relying
		      on blessings.

  * setup.py: Remove the dependency on blessings.

_frontend/status.py: Completely remove the blessings dependency from BuildStream

See merge request !808

Fix rst link formatting for guideline links

See merge request !811

This fixes an issue where the re-instantiated Source used with
Source mirroring enabled is not completely initialized.

Failing to load the ref from the project.refs file for instance,
will result in a crash at `fetch` time.

This fixes issue #666

source.py: Fix re-instantiation

Closes #666

See merge request !817

The check for directory should have always been here. The early warning
test doesn't work for `bst checkout` because caches haven't been configured
at the point of the test, so I've had to remove it.

The return value is always upload_vdir.ref.

Also add docstring to method and remove return value since it was useless

Remote exec: Remove early warning and check directory is not None

See merge request !800

Chandan Singh authored Sep 18, 2018 and Chandan Singh committed Sep 21, 2018

Currently, `source-bundle` command is entirely broken as it tries to stage the
sources in a directory that doesn't exist. Fix it by ensuring that we create
the necessary directories before calling any methods that try to use those
directories.

This fix comes with a regression test to ensure that the basic use-case
of `source-bundle` continues to work in future.

Fixes #651.

Fix source-bundle command

Closes #651

See merge request !807

setup.py: Make website the primary homepage

See merge request !795

The behavior of __assert_cached() was changed when the keystrength
parameter was introduced. This restores the previous behavior.

Fixes #607.

Fixes: 3d308894 ("element.py: Cache the result of checking whether...")

Test rebuild that requires staging of weakly cached dependency.

This is a regression test for #607.

element.py: Fix cache check in non-strict mode

Closes #607

See merge request !822

This adds directory objects to the local repository before downloading
files in the directory. However, artifact references are still stored
only after downloading the complete directory and thus, there won't be
dangling references. This will anyway be required for partial download
support.

gRPC can handle 1 MiB payloads. Increase size limit from 64 KiB to speed
up uploads.`

Use 1 MiB as payload size limit on the server side for both individual
downloads and batch uploads.

This uses BatchReadBlobs instead of individual blob download to speed up
artifact pulling, if the server supports it.

Fixes #554.

_artifactcache/cascache.py: Use BatchReadBlobs

Closes #554

See merge request !813

Also update the documentation for get_virtual_directory to make the
implications of using both methods clearer.

Stop caching virtual directories if get_directory is used.

See merge request !818

- Set default pull to not include buildtree artifact dir
- PullQueue configurable buildtree attribute
- Add 'pullbuildtrees' option to existing artifact artifactcache()
  config
- pullbuildtrees set on a per server basis from user conf
- Add --pull-buildtrees flag to bst build cli
- Add --pull-buildtrees flag to bst pull cli
- Add helper function _fetch_subdir to cascache, to fetch buildtree
  or any other subdir digest
- Make element._pull_pending not assume no need to process pull if
  artifact is cached if buildtrees are set to be pulled
- Ensure cascache.py doesn't try to checkout/extract a dangling ref

ToDo:
- Tests