Configure restricting coverage to raise or stay in a project
Problem to solve
As a team lead , I want MRs that do not keep test coverage at/above the coverage of the target branch to be blocked, so we can enforce as a team TDD/Quality/Coverage expectations in the project.
- Delaney (Development Team Lead) - Wants coverage to be increasing especially in projects with low coverage.
- Sasha (Software Developer) - doesn't want to have to add bogus tests just to appease the quality gate especially when working on a critical bug fix.
User experience goal
The user should be able to see why their pipeline can't merge when blocked by code coverage that would decrease after their merge.
Create a custom merge request approval rule similar to the security
Vulnerability Check approval rule. This would allow the user to opt their project into the requirement and also allow for an approval to override the block.
- If a project is setup to block/warn on test coverage decrease (issue), the merge widget in the Merge Request will:
- Show the
Mergebutton disabled. Users should not be able to interact with the button.
- Show a Merge request status message.
- Show a link to the documentation.
- Blocking text should say
Merge blocked: test coverage decreases with changes in this merge request. Push a new commit that increases test coverage, or refer to the [troubleshooting documentation](https://docs.gitlab.com/ee/ci/troubleshooting.html#merge-request-status-messages) to see other possible solutions.
- Show the
- Once the user fixes the code coverage issues in the MR, the status and message of the merge widget should be updated accordingly.
From the original author:
This is a really important thing for the coala community (coala-analyzer.org) as well as at least one commercial client of mine who's thinking about switching to gitlab.
Permissions and Security
- Add documentation for new keyword
- Add documentation / example for how to use in relation to coverage calculations
Availability & Testing
What does success look like, and how can we measure that?
- Documentation for usage is written
- Demonstrate turning on/off the approval rule
- Demonstrate a pipeline passing with coverage that has 0% change or increased coverage.
- Demonstrate a pipeline failing and blocking the merge because coverage decreases
- Demonstrate a pipeline failing with a coverage decrease that is overridden
What is the type of buyer?
The Team lead is most interested in this feature, this will be available in GitLab Core
Is this a cross-stage feature?