Loading CONTRIBUTING.rst +40 −12 Original line number Diff line number Diff line Loading @@ -1478,48 +1478,76 @@ Don't get lost in the docs if you don't need to, follow existing examples instea Running tests ~~~~~~~~~~~~~ To run the tests, just type:: We use `tox <https://tox.readthedocs.org/>`_ as a frontend run the tests which are implemented using `pytest <https://pytest.org/>`_. To run the tests, simply navigate to the toplevel directory of your buildstream checkout and run:: ./setup.py test tox By default, the test suite will be run against every supported python version found on your host. If you have multiple python versions installed, you may want to run tests against only one version and you can do that using the ``-e`` option when running tox:: tox -e py37 The output of all failing tests will always be printed in the summary, but if you want to observe the stdout and stderr generated by a passing test, you can pass the ``-s`` option to pytest as such:: At the toplevel. tox -- -s When debugging a test, it can be desirable to see the stdout and stderr generated by a test, to do this use the ``--addopts`` function to feed arguments to pytest as such:: .. tip:: ./setup.py test --addopts -s The ``-s`` option is `a pytest option <https://docs.pytest.org/latest/usage.html>`_. Any options specified before the ``--`` separator are consumed by ``tox``, and any options after the ``--`` separator will be passed along to pytest. You can always abort on the first failure by running:: ./setup.py test --addopts -x tox -- -x If you want to run a specific test or a group of tests, you can specify a prefix to match. E.g. if you want to run all of the frontend tests you can do:: ./setup.py test --addopts 'tests/frontend/' tox -- tests/frontend/ Specific tests can be chosen by using the :: delimeter after the test module. If you wanted to run the test_build_track test within frontend/buildtrack.py you could do:: ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track' tox -- tests/frontend/buildtrack.py::test_build_track We also have a set of slow integration tests that are disabled by default - you will notice most of them marked with SKIP in the pytest output. To run them, you can use:: ./setup.py test --addopts '--integration' tox -- --integration By default, buildstream also runs pylint on all files. Should you want to run just pylint (these checks are a lot faster), you can do so with:: ./setup.py test --addopts '-m pylint' tox -- -m pylint Alternatively, any IDE plugin that uses pytest should automatically detect the ``.pylintrc`` in the project's root directory. .. note:: While using ``tox`` is practical for developers running tests in more predictable execution environments, it is still possible to execute the test suite against a specific installation environment using pytest directly:: ./setup.py test Specific options can be passed to ``pytest`` using the ``--addopts`` option:: ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track' Adding tests ~~~~~~~~~~~~ Loading Loading
CONTRIBUTING.rst +40 −12 Original line number Diff line number Diff line Loading @@ -1478,48 +1478,76 @@ Don't get lost in the docs if you don't need to, follow existing examples instea Running tests ~~~~~~~~~~~~~ To run the tests, just type:: We use `tox <https://tox.readthedocs.org/>`_ as a frontend run the tests which are implemented using `pytest <https://pytest.org/>`_. To run the tests, simply navigate to the toplevel directory of your buildstream checkout and run:: ./setup.py test tox By default, the test suite will be run against every supported python version found on your host. If you have multiple python versions installed, you may want to run tests against only one version and you can do that using the ``-e`` option when running tox:: tox -e py37 The output of all failing tests will always be printed in the summary, but if you want to observe the stdout and stderr generated by a passing test, you can pass the ``-s`` option to pytest as such:: At the toplevel. tox -- -s When debugging a test, it can be desirable to see the stdout and stderr generated by a test, to do this use the ``--addopts`` function to feed arguments to pytest as such:: .. tip:: ./setup.py test --addopts -s The ``-s`` option is `a pytest option <https://docs.pytest.org/latest/usage.html>`_. Any options specified before the ``--`` separator are consumed by ``tox``, and any options after the ``--`` separator will be passed along to pytest. You can always abort on the first failure by running:: ./setup.py test --addopts -x tox -- -x If you want to run a specific test or a group of tests, you can specify a prefix to match. E.g. if you want to run all of the frontend tests you can do:: ./setup.py test --addopts 'tests/frontend/' tox -- tests/frontend/ Specific tests can be chosen by using the :: delimeter after the test module. If you wanted to run the test_build_track test within frontend/buildtrack.py you could do:: ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track' tox -- tests/frontend/buildtrack.py::test_build_track We also have a set of slow integration tests that are disabled by default - you will notice most of them marked with SKIP in the pytest output. To run them, you can use:: ./setup.py test --addopts '--integration' tox -- --integration By default, buildstream also runs pylint on all files. Should you want to run just pylint (these checks are a lot faster), you can do so with:: ./setup.py test --addopts '-m pylint' tox -- -m pylint Alternatively, any IDE plugin that uses pytest should automatically detect the ``.pylintrc`` in the project's root directory. .. note:: While using ``tox`` is practical for developers running tests in more predictable execution environments, it is still possible to execute the test suite against a specific installation environment using pytest directly:: ./setup.py test Specific options can be passed to ``pytest`` using the ``--addopts`` option:: ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track' Adding tests ~~~~~~~~~~~~ Loading