Next iteration of test plans, bake into requirements or MRs
We have had our test plans for a few releases now. It has done a good job of jump-starting test planning conversations for the team and managing risk for a feature. However, this is still very much external to the current process and is sometimes only worked on by an engineer on the Quality department. e.g. https://gitlab.com/gitlab-org/gitlab-ce/issues/52893
Challenges
- Test plan issues are seen as an external component to developing features
- For smaller changes, we don't really require a test plan and can be resolved in a discussion in a Merge Request
- Test plans only driven by or worked on from Quality engineers.
Goal
Revise the current test planning process. Make the process a bit more light-weight and push test planning upstream.
considerations
- Create a test planning space in the requirement template.
- Create a test planning space in the MR template that can accommodate test planning where there are no issues linked. And provide flexibility to link to a test planning section in the requirement.
Proposal
- [Mark] Update the MR template to encourage a review of existing tests, and to prompt a discussion about quality/risk guided by test design heuristics. This will change the Performance and Testing section to:
### Performance and testing
<!-- What risks does this change pose? How might it affect the quality/performance of the product?
What additional test coverage or changes to tests will be needed?
Will it require cross-browser testing?
See the test engineering process for further guidelines: https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/
If cross-browser testing is not required, please remove the relevant item, or mark it as not needed: [-] -->
- [ ] [Review and add/update tests for this feature/bug](https://docs.gitlab.com/ee/development/testing_guide/index.html). Consider [all test levels](https://docs.gitlab.com/ee/development/testing_guide/testing_levels.html). See the [Test Planning Process](https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/).
- [ ] [Tested in all supported browsers](https://docs.gitlab.com/ee/install/requirements.html#supported-web-browsers)
- [Mark] Update the feature proposal issue template to prompt a discussion about quality/risk guided by test design heuristics: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26779
- [Zeff] Update the testing guide to include a section on test design, linked to the existing test design heuristics page
- [Mark] Update the test engineering page to reflect this more lightweight test planning process in which test plan issues are entirely optional, but starting the discussion in an issue is recommended, and adding tests (typically unit tests) is expanded to reviewing/adding/updating tests at all levels: gitlab-com/www-gitlab-com!20827 (merged)
Edited by Mark Lapierre