Commit 5ec46ba8 authored by Tristan Van Berkom's avatar Tristan Van Berkom Committed by Chandan Singh
Browse files

CONTRIBUTING.rst: Updated to reflect running tests using tox.

parent 362dea64
Loading
Loading
Loading
Loading
+40 −12
Original line number Diff line number Diff line
@@ -1478,48 +1478,76 @@ Don't get lost in the docs if you don't need to, follow existing examples instea

Running tests
~~~~~~~~~~~~~
To run the tests, just type::
We use `tox <https://tox.readthedocs.org/>`_ as a frontend run the tests which
are implemented using `pytest <https://pytest.org/>`_. To run the tests, simply
navigate to the toplevel directory of your buildstream checkout and run::

  ./setup.py test
  tox

By default, the test suite will be run against every supported python version
found on your host. If you have multiple python versions installed, you may
want to run tests against only one version and you can do that using the ``-e``
option when running tox::

  tox -e py37

The output of all failing tests will always be printed in the summary, but
if you want to observe the stdout and stderr generated by a passing test,
you can pass the ``-s`` option to pytest as such::

At the toplevel.
  tox -- -s

When debugging a test, it can be desirable to see the stdout
and stderr generated by a test, to do this use the ``--addopts``
function to feed arguments to pytest as such::
.. tip::

  ./setup.py test --addopts -s
   The ``-s`` option is `a pytest option <https://docs.pytest.org/latest/usage.html>`_.

   Any options specified before the ``--`` separator are consumed by ``tox``,
   and any options after the ``--`` separator will be passed along to pytest.

You can always abort on the first failure by running::

  ./setup.py test --addopts -x
  tox -- -x

If you want to run a specific test or a group of tests, you
can specify a prefix to match. E.g. if you want to run all of
the frontend tests you can do::

  ./setup.py test --addopts 'tests/frontend/'
  tox -- tests/frontend/

Specific tests can be chosen by using the :: delimeter after the test module.
If you wanted to run the test_build_track test within frontend/buildtrack.py you could do::

  ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track'
  tox -- tests/frontend/buildtrack.py::test_build_track

We also have a set of slow integration tests that are disabled by
default - you will notice most of them marked with SKIP in the pytest
output. To run them, you can use::

  ./setup.py test --addopts '--integration'
  tox -- --integration

By default, buildstream also runs pylint on all files. Should you want
to run just pylint (these checks are a lot faster), you can do so
with::

  ./setup.py test --addopts '-m pylint'
  tox -- -m pylint

Alternatively, any IDE plugin that uses pytest should automatically
detect the ``.pylintrc`` in the project's root directory.

.. note::

   While using ``tox`` is practical for developers running tests in
   more predictable execution environments, it is still possible to
   execute the test suite against a specific installation environment
   using pytest directly::

     ./setup.py test

   Specific options can be passed to ``pytest`` using the ``--addopts``
   option::

     ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track'


Adding tests
~~~~~~~~~~~~