Add support for running tests in BuildStream
Background
From the discussion on the mailing list:
As software integrators, we would like to run tests as part of BuildStream builds and be able to validate (at a later stage in build) that the tests for a given element have passed. Ideally, such tests should have the following properties:
1.) Tests should not block reverse dependency builds.
2.) Strong association between an element and its tests.
3.) Additional dependencies for testing an element should not be required while building it.
4.) Elements should be able to assert that the tests for their dependencies have passed.
5.) Test elements are not necessarily pass-through elements - they can have meaningful output such as coverage reports.
Task description
The proposed solution is to introduce the concepts of 'conditions' and 'abstract dependencies'. For more detail, see the full proposal from the mailing list
-
Add a new conditionsfield to the bst element file with subfieldsprovidesandrequires, e.g.
conditions:
provides:
tested:
- app-test.bst
packaged:
- app-pkg.bst
requires:
- tested
-
Add abstract dependencies (#777) -
Add functionality to bst showto check theconditionsfields of an element (for possible sample outputs, see the examples under part 3 of the mailing list proposal) -
Expose a Python API for staging an element's build tree (see part 4 of the mailing list proposal for the justification for this) -
Provide a method to guarantee that the build tree we want to stage (for the purposes of running tests) exists in the dependency artifact
Acceptance Criteria
Users can run tests as part of the build of an element such that the test properties outlined above are satisfied.