Use -- tag reliable for the regression suite in addition to smoke tests that will guard the release
Per work done in our Rapid Action:
- Create another set of tests using
reliable
tag and include all the end-to-end tests for devopsplan https://gitlab.com/gitlab-org/quality/team-tasks/issues/337 - Add :reliable test documentation and processes https://gitlab.com/gitlab-org/quality/team-tasks/issues/378
We now have a first set of reliable tests via -- tag reliable
. We need to start enabling this in the release process. Currently we have 36 of these tests which is trade-off between smoke and all tests.
An example on how this new sets of test can be run.
bin/qa Test::Instance::All http://localhost:3000 -- qa/specs/features/ --tag reliable
Edited by Walmyr Lima e Silva Filho