Test plan for Reviews: batch comments on merge requests
Test Plan
Introduction
This test plan is for https://gitlab.com/gitlab-org/gitlab-ee/issues/1984, which adds the ability to add comments to a batch and submit them all at once.
We're going to call the batch commenting feature: “review”. A review is a group of comments that are published together. You can start a review, add a comment to a review, discard a review, and finish/submit a review.
Scope
- The review feature is only available in merge requests, when creating a discussion from a code line, or replying to a discussion (from a code line or from the “Discussion” tab) — it is not available when adding a comment/discussion in the “Discussion” tab.
- Sending a single email for a batch of comments is a separate issue: https://gitlab.com/gitlab-org/gitlab-ee/issues/4326
- The content of comments is out of scope since this change doesn't affect the comment entry component
ACC Matrix
The matrix below identifies the Attributes, Components, and Capabilities relevant to the scope of this test plan.
Attributes (columns) are adverbs or adjectives that describe (at a high level) the qualities testing is meant to ensure Components have.
Components (rows) are nouns that define major parts of the product being tested.
Capabilities link Attributes and Components. They are what the product needs to do to make sure a Component fulfills an Attribute.
For more information see the Google Testing Blog article about the 10 minute test plan and this wiki page from an open-source tool that implements the ACC model.
Simple | Responsive | Intuitive | Reliable | |
---|---|---|---|---|
MRs | 4 | 1 | 2 | 6 |
Capabilities
- MRs are:
- Simple
- It's easy to start/continue/complete a review
- It's easy to resolve discussions
- It's easy to add individual comments, including when a review is pending
- Reviews are only visible to the reviewer - other users can't see reviews until they are submitted
- Responsive
- The results of actions are displayed promptly regardless of the number of comments or the length of the discussion or the size of the MR.
- Intuitive
- It's clear which reviews and comments belong to which discussions
- Review comments are always presented at the end of the list - new saved comments are inserted above review comments
- Reliable
- Starting/completing/discarding a review updates the number of discussions appropriately
- Discussions that are marked resolved when a review is submitted can be marked unresolved
- Reviews are included when using the button to 'Jump to next unresolved discussion'
- The button to resolve a discussion in a new issue works for reviews
- Previous reviews can still be accessed if a new commit changes the relevant line of code
- Reviews are persisted for the user, not the browser or session
- Simple
Test Plan
This feature is simple enough that the set of Capabilities are sufficient to guide testing. Note that some of the Capabilities don't lend themselves to automation, however the aim is to automate as much as possible, at testing levels that are as low as is reasonable.
There is a feature test that includes many relevant examples for comments and resolving discussions that could be extended to cover this change. There are some more feature tests related to comments on merge requests.
It would be worthwhile having at least one end-to-end test that involves this feature. One possibility is checking that a review can be completed if a force push removes the line of code that the review was started on.