Valentin David authored Aug 14, 2018 and Valentin David committed Aug 14, 2018

We now delay refreshing submodules until we have a clone, that is after
we have fetched the main fetcher.

Fixes #537

Delay refreshing git submodule until we have a clone

Closes #537

See merge request !656

Qinusty authored Aug 15, 2018 and Qinusty committed Aug 15, 2018

This should avoid current issues accessing the necessary sources which
are blocking CI currently.

Fix CI - ftp.gnu.org unreachable

See merge request !665

Qinusty authored Aug 03, 2018 and Qinusty committed Aug 15, 2018

This allows for users to configure fatal-warnings to be either a list of
warnings.

This commit deprecates the use of fail-on-overlap within project.conf,
this will now use the fatal-warnings configuration item.

element.py: Cache key calculation now takes into account all of the
fatal-warnings

tests: This modifys the tests/frontend/overlaps.py tests to support the
new fatal-warnings configuration. Backwards compatibility is also
tested for `fail-on-overlap`

_versions.py: BST_FORMAT_VERSION bumped to 15 for fatal-warnings
	      BST_CORE_ARTIFACT_VERSION bumpted to 5 for fatal-warnings

Fixes: #526

Qinusty authored Aug 09, 2018 and Qinusty committed Aug 15, 2018

This follows the implementation of configurable warnings.

Qinusty authored Aug 08, 2018 and Qinusty committed Aug 15, 2018

This includes detailing the use of `fatal-warnings` within project.conf

Qinusty authored Aug 09, 2018 and Qinusty committed Aug 15, 2018

This adds multiple tests for custom plugin warnings and core warnings,
providing checks for both cases which should cause warnings and errors
when configured as fatal.

Also adds tests for cache key calculations.

Configurable Warnings

Closes #526

See merge request !627

Continue to accept requests from non-conforming BuildStream clients for
the time being to ease migration.

This requires an updated server.

Fixes #572.

Replace assertions with gRPC error responses.

CAS: Fix resource_name format for blobs

Closes #572

See merge request !660

CAS-backed virtual directory implementation

See merge request !481

Source plugin implementations can now specify that they need access to
previously staged sources by specifying
`BST_REQUIRES_PREVIOUS_SOURCES_TRACK` and/or
`BST_REQUIRES_PREVIOUS_SOURCES_FETCH`, corresponding to access at `track`
and `fetch` times respectively.

Fixes #381.

Replaces !505. For relevant discussion, see this discussion:
!505 (comment 83780747)

`pip` source plugin can stage python packages that are either specified
directly in the element definition or picked up from `requirements.txt`
from previous sources. In order to support the latter use-case
(which is also the primary motivation for this plugin), this plugin
requires access to previous sources and hence is an example of a
Source Transform source.

Also, bump `BST_FORMAT_VERSION` as this patch adds a new core plugin.

Allow source plugins to access previous sources

Closes #381

See merge request !568

Valentin David authored Aug 14, 2018 and Tristan Van Berkom committed Aug 16, 2018

Ostree mirrors were not sharing the same local repository, so it was
impossible the request refs from the right local repository when data
was fetched from a mirror rather than upstream.

Instead of having several repository with one remote each, we now
have one repository with several remotes.

This fixes #538.

Fix ostree repository mirroring

Closes #538

See merge request !658

This patch mitigates the granularity of the mtimes used on the default
gitlab runners, Allowing the test suite to pass on these runners.

Trying to mitigate a file system issue

See merge request !595

https seems broken on the Debian image:

START   autotools/hello/42930621-fetch.499.log
START   Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
FAILURE Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
FAILURE tar source at hello.bst [line 16 column 2]: Error mirroring https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:720)>

Use http instead of https for the gnu ftp mirror

See merge request !676

os.errno does no longer work with Python 3.7

Closes #577.

cascache: use errno module

Closes #577

See merge request !674

Tristan Van Berkom authored Aug 16, 2018 and Tristan Van Berkom committed Aug 17, 2018

Use the optionally implemented desktop notification method
to notify when a session completes.

Previously, this used to only notify when an interactive prompt
appears.

This is an attempt to fix #385

_frontend/app.py: Notify session completions

Closes #385

See merge request !672

Raise a error at configure time if the track and ref properties are
not present in the sources.

This is to address #471
that documented unhelpful behaviour when tracking git sources. However
the issue was also identified in ostree.

Updating the no track or ref fix to match the buildstream style

See merge request !628

Qinusty authored Aug 14, 2018 and Qinusty committed Aug 17, 2018

Fixes #531: Jobs were retrying when terminated, this lead to the process
being spawned again and starting up fresh.

Prevent jobs retrying on terminate

Closes #531

See merge request !662

This is causing pytest to try to import the recently added app1.py
from the pip related tests. This fails because it's not meant to be
imported outside of the sandbox environment.

setup.cfg: Add tests/integration/project to norecursedirs

See merge request !682

It would have been more convenient to store the quota size in the
ArtifactCache, except that the scheduler needs the quota size and
doesn't have access to the ArtifactCache.

Jonathan/cache cache size

See merge request !679

Jonathan/cascache cell var from loop

See merge request !686

Valentin David authored Aug 08, 2018 and Qinusty committed Aug 21, 2018

Issue was introduced by 171e803f (include directive) and the fix was
found courtesy of @Qinusty. This fixes also the include
feature. Because elements are to be serialized, the included fragments
need to use copy_tree when loaded.

Related to #470.

Fix broken indentation after track

See merge request !622

Pipeline: Skip except_elements logic if no elements to except

See merge request !688

Chandan Singh authored Aug 21, 2018 and Tristan Van Berkom committed Aug 21, 2018

These new images come with pinned versions of `pytest` and `pylint`:

- `pylint`: `== 2.1.1`
- `pytest`: `>= 3.7`

Chandan Singh authored Aug 21, 2018 and Tristan Van Berkom committed Aug 21, 2018

These packages should only be updated as a deliberate change and the
code should be tested with the new versions to ensure that the test
still pass.

Any future updates should also happen in sync with the
buildstream-docker-images repository so that the testsuite base images
also enforce the same restrictions for the development requirements.

Replaces !684.

Restrict pylint/pytest versions and bump CI tags

See merge request !689

Qinusty authored Aug 21, 2018 and Qinusty committed Aug 21, 2018

Providing warn with detail=None (default) will no longer print None when
a fatal warning is triggered.

Fatal warnings no longer print None when detail=None

See merge request !694

Fill out those fields in `setup.py` that are required for publishing a
package on PyPi.

Part of #587.

setup.py: Add fields required for publishing on PyPi

See merge request !695

Now we allow the notification to happen on any TERM
which starts with 'xterm' or 'vte', and we only do it
if the VTE_VERSION is >= 4600, where we know for sure that
VTE will not print garbage on the terminal.

Fixes #385

_frontend/linuxapp.py: Fix special casing around desktop notification escape sequence

Closes #385

See merge request !692

This patch just displays a better message than the default stack trace
but dose not try to fix the problem. A further patch will be created but
it effects versioneer so may take longer to land as it may need to go
via versioneer mainline.

This has been done in a way that I hope might be able to be upstreamed
in to versioneer. This is not garanteed but it would be good if it
could be.

Please see https://github.com/warner/python-versioneer/pull/186 and
https://github.com/wsalmonct/python-versioneer/tree/feature/regexTags
for further details.

Search for tags with the *.*.* patten for version

See merge request !601

When GDB looks for debug symbols it looks for the full path of the file
inside the configured debug-file-directory. For example, if the
debug-file-directory is set to a default of /usr/lib/debug, and you are
debugging /usr/bin/python3, GDB will look for its debug symbols at this
location: /usr/lib/debug/usr/bin/python3.

BuildStream has been putting all debug files inside /usr/lib/debug
under their $(basename), so in the above example GDB would fail to find
any debug symbols for /usr/bin/python3 because they would be in the
incorrect locatoin of /usr/lib/debug/python3.

Don't strip pathname from debug symbol files

See merge request !697

Currently, running any setuptools commands using `setup.py` generates a
warning like so:
        warning: manifest_maker: MANIFEST.in, line 21: unknown action 'dev-requirements.txt'

The syntax is invalid and got accidentally introduced in !637 but it's a
shame that this is not a warning and no obvious way to make it an error
if there are invalid things in MANIFEST.in file.

MANIFEST.in: Fix include for dev-requirements.txt

See merge request !704

When fixing terminal notifications, I had introduced a bug
with accesses to `os.environ` which triggered KeyError, this
patch fixes it.

Valentin David authored Aug 22, 2018 and Tristan Van Berkom committed Aug 23, 2018

Fixes #576.

Improve error message for deleted open workspaces

Closes #576

See merge request !703

Valentin David authored Aug 20, 2018 and Tristan Van Berkom committed Aug 23, 2018

Fixes #561

Remove artifact extracts when artifact expires in cache

Closes #561

See merge request !685

Buildstream uses the Terminal.does_styling attribute, which was only
added in blessings 1.6.

Blessings helpfully returns an empty string when calling an nonexistent
attribute on the Terminal object, instead of failing.

Because Fedora has blessings 1.5, Buildstream thought that my terminal
couldn't handle styling, and just didn't print the status bar, silently.

Specifying the minimum version avoids this silent failure.