Add `.in` and `.txt` requirements files for BuildStream's pure python
dependencies. For each pair, the `.in` file is supposed to capture the
loose version requirements, and the corresponding `.txt` file is
supposed to have frozen requirements. We have 3 such sets:

* `requirements`: BuildStream's runtime dependencies
* `dev-requirements`: Dependencies for running tests
* `plugin-requirements`: Dependencies for core plugins

Note that the frozen requirements files will only be used for testing
purposes, and `setup.py` will continue to read loose requirements.

Add `tox.ini` file that will enable us to use
[tox](https://tox.readthedocs.io/) as a frontend for running tests.
Since we share the config via `setup.cfg` and requirements via
requirements files, commands like `python3 setup.py test` will continue
to work.

Instead of invoking tests throung `setup.py`, use `tox` as a frontend in
the CI pipelines.

Since we don't allow use of site packages by default in our `tox`
configuration, people will need to install non-python build dependencies
of some of our dependencies that do not provide pre-built wheels. We
have two such packages at the moment:

* psuitl: requires python C headers, compiler
* pygobject: requires python C headers, pkg-config, compiler, cairo
  headers, and gobject libraries

These new `.in` and `.txt` are making the repository look very
cluttered. Move them to a separate `tools` directory to make it look a
bit cleaner.

Currently the CI and the docs both have to duplicate the same inforation
about how to gather dependencies etc, and have to use hacky ways to run
them.

Add a new `docs` environment to our tox setup so that building docs is
as simple as running `tox -e docs`.

When we update our dependencies, developers will eventually need to
recreate their `tox` environments to get correct results. This happens
because `tox` isn't particularly good at recognizing changes in the
requirements files.

Use tox as a frontend for running tests

See merge request !1027

Chandan Singh authored Jan 03, 2019 and Chandan Singh committed Jan 03, 2019

In !1028, we added a test specifically to test
that BuildStream correctly raises an warning when the name of an element
contains characters that are invalid on Windows. Unfortunately, we
didn't see it coming that it would make it impossible to checkout this
branch on Windows.

Fix it by generating this file, only if we are not running on Windows.

* tests/testutils/site.py: Add `IS_WINDOWS` check
* tests/frontend/buildcheckout.py: Generate file with invalid filename
  on the fly
* Remove tests/frontend/project/elements/invalid-chars.

Fixes #842.

Note that this may still cause issues on WSL when running tests on a
shared filesystem, but that seems to be a generic issue on WSL with
`os.rename`.