Line-by-Line code coverage results displayed in a Merge Request's changes tab.
Problem to solve
Does this merge request include tests that cover all the lines of code changed in it?
Approvers for a merge request want to easily see whether tests cover every line newly changed in a pull request before it merges.
Intended users
Developers, Managers, Leads can quickly establish that new code submitted in a pull request is improving the total project coverage before approving a merge.
Further details
- there are many different code coverage tools
- they all report file, line number, and visit count
- these tools may run inside a CI/CD pipeline or separately
Proposal
Since different programming languages use different code coverage tools, we would want GitLab to provide an API for reporting whether a line in a diff (associated with a commit) was covered or not by a testing suite. This information would be sent to GitLab via the same API authN/Z that the 3rd party pipeline integration is currently implemented over. This information may or may not be sent so its absence should not break anything. It's later arrival via a special tool should not be a blocker.
- when no coverage data is supplied, do nothing different.
- report covered lines with a positive mark (green highlight under the text) on just the line numbers matched by a code coverage measurement tool
- a simple API reports commit SHA, file name, line number, covered in a single document. Coverage information should apply wherever the lines for a commit are shown.
This strategy will require writing adapters for various code-coverage tools to work with the API.
Permissions and Security
This should be an extension of the CI/CD integration APIs that allow 3rd party CI/CD integrations.
Documentation
Testing
What does success look like, and how can we measure that?
Merge request show which lines are covered by test. This information can be reviewed line-by-line later by technical managers, leads, and developers auditing code quality.
What is the type of buyer?
Technical Manager, Software Engineering Team Lead, Technical Team Lead, Software Development Director, Development Lead, trying to enforce code quality per merge request.