<!-- The first section "Release notes" is required if you want to have your release post blog MR auto generated. Currently in BETA, details on the **release post item generator** can be found in the handbook: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator and this video: https://www.youtube.com/watch?v=rfn9ebgTwKg. The next four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended in your first draft, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Note about availability Please note that this new MR rule will not be available until https://gitlab.com/gitlab-org/gitlab/-/issues/331001 is complete. ### Release notes If you and your team want to enforce that the code coverage of your project can not decrease from a change you have had to come up with create solutions to check how the coverage changes and fail a pipeline if it decreases. While this was possible within GitLab for teams already used to [Required Approvals](ls/#required-approvals) this creates multiple workflows within the same Merge Request. Now you can create an approval rule to enforce that code coverage does not decrease as part of a merge request so this workflow follows what the team already does. <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> ### Problem to solve <!-- What problem do we solve? Try to define the who/what/why of the opportunity as a user story. For example, "As a (who), I want (what), so I can (why/value)." --> As a team lead , I want MRs that do not keep test coverage at/above the coverage of the target branch to require approval, so we can enforce as a team TDD/Quality/Coverage expectations in the project. Users can fail the pipeline today by checking coverage in a separate job and failing the pipeline as shown in this [Unfiltered recording](https://www.youtube.com/watch?v=XjcUqZu0w5o). ### Intended users * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) - Wants coverage to be increasing especially in projects with low coverage. * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) - doesn't want to have to add bogus tests just to appease the quality gate especially when working on a critical bug fix. ### User experience goal <!-- What is the single user experience workflow this problem addresses? For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>" https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/ --> The user should be able to see that their MR requires approval because the code coverage would decrease the project coverage. ### Proposal Create a custom [merge request approval rule](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) similar to the security `Vulnerability Check` approval rule. This would allow the user to opt their project into the requirement and also allow for an approval to override the block. * If a project is setup with the MR approval rule when the MR would decrease coverage: * Show the `Merge` button disabled, following the same logic as when required approvals are missing. Users should not be able to interact with the button. * Once the user fixes the code coverage issues in the MR OR there is approval to override, the MR can be merged. #### ~backend Here a [POC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59698) of how a potential implementation will look like (need to be updated as it's currently relying on an existing coverage report artifact). We will add a **new merge request approval rule for code coverage** and compute the percentage coverage at the pipeline level as described below. ##### First iteration **No support for jobs with coverage at the project level.** For this first iteration of this feature we will only rely on jobs using the [coverage keyword from a gitlab-ci.yml configuration file](https://docs.gitlab.com/ee/ci/yaml/#coverage). This will allow us to determine easily without ~performance concern which builds we need to fetch the coverage for without dealing with unfinished builds. We would need to parse and update the coverage data in our new service for both `base_report` and `head_report`: 1. Fetch builds with a [coverage_regex](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/structure.sql?expanded=true&viewer=simple#L10497) - [populated on build creation](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/yaml_processor/result.rb#L71). 2. Update the coverage data for those builds - [builds#update_coverage](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/build.rb#L604-607) 3. Determine pipeline coverage percentage - [pipeline#coverage](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline.rb#L635-640) ```mermaid graph LR subgraph "UpdateApprovalRulesWorker" Build1[FetchBuildsWithCoverageRegex] --> Build2[BuildsUpdateCoverage] Build2[PipelineCoverage] --> Build3[GetCoveragePercentage] end ``` ### Further details From the original author: > This is a really important thing for the coala community (coala-analyzer.org) as well as at least one commercial client of mine who's thinking about switching to gitlab. > <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> ### Permissions and Security N/A ### Documentation * Add documentation / example for how to setup a coverage Merge Request Rule ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? #### Acceptance Criteria * Documentation for usage is written * Demonstrate turning on/off the approval rule * Demonstrate a pipeline passing with coverage that has 0% change or increased coverage. * Demonstrate a pipeline failing and blocking the merge because coverage decreases * Demonstrate a pipeline failing with a coverage decrease that is overridden <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. Create tracking issue using the the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md --> ### What is the type of buyer? The Team lead is most interested in this feature to enforce team/company norms. As this leverages Merge Request Rules this will be available in ~"GitLab Premium" and above. ### Is this a cross-stage feature? No. ### Links / references