Angelos Evripiotis authored Nov 26, 2018 and Angelos Evripiotis committed Dec 13, 2018

Update the section on adding tests, to fulfill these broad points:

- Mandate end-to-end testing, with rationale.
- Mention internal APIs as endpoints for testing.
- Warn against pitfalls of unit-testing.

It's more text than I would have liked, perhaps in later work we'll be
able to say it with less.

This change integrates feedback from the mailing list
'Guidance on Unit Tests' thread:
https://mail.gnome.org/archives/buildstream-list/2018-November/msg00045.html

contributing: more clarity on testing

See merge request !973

Following the message thread
https://mail.gnome.org/archives/buildstream-list/2018-November/msg00106.html,
implement a new command group called `source`. Move existing `track`,
`fetch`, and the recently added `source-checkout` commands under this
group.

For `track` and `fetch`, this is a BREAKING change, as the old commands
have been marked as obsolete. Using them will result in an error message
that refers people to use the new versions, like `bst source fetch`
instead of old `bst fetch`. `source-checkout` will now become
`source checkout` (the dash has turned into a space), and is not a
breaking change as it was added in the current development cycle.

Note that the functionality to hide commands from help output was added
only recently in Click, so the minimum version of Click that we now
require is 7.0.

Summary of changes:

* _frontend/cli.py: Add `source` command group, mark previous versions
  as obsolete and hide them from the help output.
* _frontend/complete.py: Fix completion for hidden commands.
* setup.py: Bump Click minimum version to 7.0.
* tests: Update to cope with the new command names.

Fixes #814.

Also, change the existing entry about `source-checkout` to be
`source checkout` instead. And, while we are there, move it near the
other announcements about the source command group.

Introduce new "source" command group

Closes #814

See merge request !1003

This is used when sending execution requests, to specify which instance of the
execution server to use.

Partial fix for #627.

Adds instance name support for the remote execution storage service.

Adds the 'instance_name' parameter, which may be None, to most GRPC
calls in the CASCache object. ByteStream requests already have
instance_name supplied in the resource name, so do not need the
parameter.

Closes #627.

Remote-execution instance configuration support

Closes #627

See merge request !952

Fixes issue #740

Shell --build has optional buildtrees

Closes #740

See merge request !986

README.rst: Add license badge

See merge request !1014

The queue shouldn't need to know about individual sources. This is in
line with _track() and _get_consistency().

Do not call fetch() for cached sources

See merge request !992

Add a `_GitSourceBase` class to act as the base class for deriving
source plugins that work with Git. The first user of this base class is
the `git` source plugin that's in core at the moment.

The contents of this base class is almost identical to the existing `GitSource`
class. The two notable differences are:

- the private methods now have leading underscores, as some were missing
  it originally
- the name of the class

Note that we are exposing a private member here as we expect it to move to a
separate package soon. See the following discussion for more details:
#739 (comment 124819869)

Fixes #739.

Expose base class for Git source plugins

Closes #739

See merge request !1019

Angelos Evripiotis authored Dec 19, 2018 and Angelos Evripiotis committed Dec 20, 2018

Now that we are also looking for WORKSPACE_PROJECT_FILE, we should add
it to the error message. While we're there, also mention the directory
we are looking in - this could help scripters be clear about what went
wrong.

The new error message looks like this:

    Error loading project: None of ['project.conf', '.bstproject.yaml']
    found in '/src/temp/blah' or any of its parent directories

Angelos Evripiotis authored Dec 19, 2018 and Angelos Evripiotis committed Dec 20, 2018

In the event that the project could not be found, stop BuildStream from
asking if the user would like to create a new project. Exit with error
instead, and give a hint to the user in case they're new.

As proposed on the mailing list here:
https://mail.gnome.org/archives/buildstream-list/2018-December/msg00082.html

The new interaction looks like this:

    $ bst show nonsuch.bst
    No project found. You can create a new project like so:

        bst init

    Error loading project: None of ['project.conf', '.bstproject.yaml']
    found in '/src/temp/blah' or any of its parent directories

Fixes #826

BREAK: remove auto-init behaviour

Closes #826

See merge request !1015

Chandan Singh authored Dec 14, 2018 and Chandan Singh committed Dec 20, 2018

Use raw strings for regex searches, which is the preferred way to do
regular expressions in Python.

Without this patch, currently we get the following warnings:

```
tests/frontend/logging.py:44
  /builds/BuildStream/buildstream/dist/buildstream/tests/frontend/logging.py:44: DeprecationWarning: invalid escape sequence \[
    m = re.search("\[\d\d:\d\d:\d\d\]\[\]\[\] SUCCESS Checking sources", result.stderr)

tests/frontend/logging.py:80
  /builds/BuildStream/buildstream/dist/buildstream/tests/frontend/logging.py:80: DeprecationWarning: invalid escape sequence \d
    m = re.search("\d\d:\d\d:\d\d,\d\d:\d\d:\d\d.\d{6},\d\d:\d\d:\d\d,,,SUCCESS,Checking sources", result.stderr)
```

Chandan Singh authored Dec 14, 2018 and Chandan Singh committed Dec 20, 2018

Specify flags at the start of the expression as per the recommendation
of the standard library.

Without this patch, we currently get the following warning:

```
tests/examples/junctions.py::test_open_cross_junction_workspace
  /builds/BuildStream/buildstream/dist/buildstream/buildstream/utils.py:213: DeprecationWarning: Flags not at the start of the expression '\\/[^/]*\\Z(?ms)'
    regexer = re.compile(expression)
```

Fix Deprecation warnings from regex module

See merge request !1010

_stream.py: fix _buildtree_pull_required logic

See merge request !1021

Recently we have been adding some tests which use the new tmp_path
fixture from pytest, this is new in 3.9.

Since Thomas Nixon fixed the datafiles plugin to be compatible
with pytest >= 3.8.0, we can now depend on it and avoid the
warnings and errors which the older datafiles plugin causes.

This fixes issue #636

After having fixed the more recent linting errors, the
tests should pass with recent versions of pylint. Tested
with pylint 2.2.2.

Misc fixes to the tests and newer linter errors

Closes #636

See merge request !1011

The python netrc module will raise OSError in the case
that HOME is not set, this was discovered while running
tests under tox.

Treat None values in the passed dictionary as keys to be removed
from the environment, this was already happening at restoration time.

This happens when the netrc module is searching for a ~/.netrc
file and it doesnt find any HOME set.

Fix netrc crasher

See merge request !1025

Better to use the same API everywhere for thecking CLI results.

Junction test addition

See merge request !1029

Chandan Singh authored Dec 30, 2018 and Tristan Van Berkom committed Dec 31, 2018

Currently we some duplication in the way we check for invalid filenames.
To make it more robust and allow room for adding more warnings, refactor
it into a separate method `_warn_invalid_elements()` that handles just
this.

Chandan Singh authored Dec 30, 2018 and Tristan Van Berkom committed Dec 31, 2018

See https://mail.gnome.org/archives/buildstream-list/2018-December/msg00061.html
for some related discussion.

Add warnings about invalid characters in filename

See merge request !1028

Chandan Singh authored Dec 29, 2018 and Chandan Singh committed Jan 02, 2019

https://github.com/sphinx-doc/sphinx/issues/4375 has been closed as
`wontfix`. Update the comment in doc/Makefile that mentions it, based on
the discussion on the issue.

doc/Makefile: Update comment about sphinx entrypoint

See merge request !1026

We need to bump the testsuite image versions to use the new
pycodestyle update that allows excluding file paths, so that we can
exclude generated python code.

Javier Jardón authored Aug 10, 2018 and Tristan Maat committed Jan 02, 2019

pep8 package was renamed to pycodestyle to reduce confusion.

Disabled warnings:
W504 - Line break after binary operator (seems people like this)
W605 - Invalid escape sequence (some of our regexes use these)

Use pycodestyle instead pep8 python module

See merge request !638

This change is related to #222

This change is related to #222