The CAS uses a temp directory while manipulating the cache, temporary
files can be added and removed while adding artifacts to the cache here.

Since calculation of the cache size happens in parallel to artifact
cache additions, this causes race conditions in the size calculation
job, as we end up calling `stat` on temporary files which are being
removed in parallel.

Handle this by moving the temporary directory out of the way, and avoid
considering the tmp directory when calculating the cache size

Fix race condition when calculating disk usage

See merge request !600

When open(2) is used with flags O_CREAT|O_RDWR, the file descriptor
must be readable. Unfortunately O_RDWR was not passed which made
read fail with EBADF and mmap to signal SIGBUS.

This issue happened with man-db for example.

Fixes #143.

Keep original flags for create in SafeHardlinks.

Closes #143

See merge request !624

Multiprocessing with n_jobs given by multiprocessing.cpu_count() is not optimal for
systems where not all CPU can be used (in particular, Docker, CI etc).

As suggested in the multiprocessing docs
(http://docs.python.org/3/library/multiprocessing.html#multiprocessing.cpu_count)
using len(os.sched_getaffinity(0)) might be better.

even if the machine have mores cores available

Patch taken from YBD: https://gitlab.com/baserock/ybd/blob/master/ybd/app.py#L227

buildstream/_project.py: Restrict max-jobs

See merge request !620

Pin ruamel.yaml version to <= 0.15

See merge request !630

This fixes #191

A note has been added to NEWS explaining backwards
compatibility issues

Patch for issue #191 support relative workspaces

Closes #191

See merge request !504

Valentin David authored Jul 25, 2018 and Valentin David committed Aug 10, 2018

Fixes #498

Set environment in bwrap command line instead of its environment

Closes #498

See merge request !565

In some cases, such as when working inside a virtual environment, it can
be desirable to install all dependencies for running tests using `pip`.
This is currently not possible since setuptools does not support
installing these dependencies in a virtual environment (by design).
(See https://stackoverflow.com/a/21003259.)

To circumvent this issue, move such requirements to
`dev-requirements.txt` file that can be used easily with
`pip install -r`. This also enables tests to be run directly using
`pytest`, which can be more convenient than `-addopts` approach when one
needs to add multiple options.

This will also be useful in creating better testuite images, and fix
some of the issues noticed in
buildstream-docker-images!56.

Move development reqirements to dev-requirements.txt

See merge request !637

Fix typo in dev-requirements.txt

See merge request !641

.gitlab-ci-yml: Add ubuntu 18.04 test

See merge request !523

This also fixes an error with pylint in setup.py:
setup.py:226:19: R1718: Consider using a set comprehension (consider-using-set-comprehension)

Add job to run tests in fedora 28

See merge request !643

Valentin David authored Aug 06, 2018 and Tristan Van Berkom committed Aug 12, 2018

Instead of copying metadata on files staged by local, we manually set
mode to 0755 or 0644 depending on whether user execution was enabled
on source file.

This makes file modes deterministic independently on the way source
was distributed.

Non-deterministic mode copying all metadata can still be enabled by
disable 'deterministic' Boolean configuration on the plugin.

Fixes #527.

Valentin David authored Aug 06, 2018 and Tristan Van Berkom committed Aug 12, 2018

This fix is applied to plugins bzr, git, patch.

Fixes #543 #544 #555.

Valentin David authored Aug 07, 2018 and Tristan Van Berkom committed Aug 12, 2018

Staging could end-up with file with different rights depending on the
umask. The extracted files need to get their access rights fixed.

Deterministic staging

Closes #543, #544, #555, and #527

See merge request !616

Chandan Singh authored Aug 12, 2018 and Javier Jardón committed Aug 12, 2018

`python3-arpy` package is not available on Fedora so it has to be
installed using `pip`. Update instructions accordingly.

For reference, here is a pipeline that failed when trying to install
`arpy` using current instructions:
https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/88553245.

doc: Fix install instructions for Fedora

See merge request !640

Add cachekey tests for the recently added `remote` source plugin to
ensure that future changes do not break API compatibility.

remote source: Add cachekey test

See merge request !636

Valentin David authored Aug 08, 2018 and Tristan Van Berkom committed Aug 13, 2018

Stream._load() now returns early without resolving build pipeline when
only tracking. Resolving track pipelines does not require to fully
load project configurations when when elements to track are only
junctions.

However build pipelines require to fully load project
configurations. This might not be possible in the case a project
configuration includes a file from a junction that yet needs to be
tracked.

Fixes #565.

Fix tracking of junctions used in project.conf

Closes #565

See merge request !619