Skip to content

Support Code Coverage Badge for Successful Job without Successful Pipeline

Problem to solve

Currently, to have the code coverage badge successfully display, your project needs to have at least one successful pipeline. This is limiting to users that have blocking manual deploy jobs, as their pipelines don't always complete successfully. This feature request proposes code coverage badges for jobs that don't rely on a successful pipeline.

For example, a user has the following code coverage badge on their project's README.md:

![coverage](https://gitlab.com/gitlab-org/gitlab-ce/badges/master/coverage.svg?job=coverage)

Their coverage job successfully completes and they have a coverage percentage for that job. However, their pipeline does not successfully complete because there are blocking manual deploy tasks. They see coverage: unknown on the badge.

From the customer:

If I’m pointing the badge to a job, the job has a coverage value on it then report that value for the master branch. Coverage values should reflect the state of the branch code. Coverage has nothing to do with deployment, versions or packaging state (I’d be interested to hear a valid use case otherwise).

Customer ticket --> https://gitlab.zendesk.com/agent/tickets/113248 (internal use only)

Target audience

Further details

This would increase the availability of the code coverage badge to customers that have blocking manual jobs in their pipelines.

Proposal

The code coverage report code base requires that a successful pipeline be present in the project. This issue changes to getting the coverage value from the latest pipeline that has a successful coverage job regardless of pipeline status. For cases of jobs that summarize coverage values from various jobs we assume they will utilize depends and nothing will need to change for those users.

Latest Coverage Job State Latest Pipeline State What to display on badge
Pass Pass This pipeline's coverage job value
Pass Fail This pipeline's coverage job value, this is a change
Pass Not Finished This pipeline's coverage job value, this is a change
Fail Pass The last successful coverage job value, this is a change
Fail Fail The last successful coverage job value, this is a change
Fail Not Finished The last successful coverage job value, this is a change
Not run in pipeline Pass/Fail/Not Finished The last successful coverage job value, this is a change
Never Ran Pass/Fail/Not Finished Show unknown, no change

A future iteration of this may be to make it possible to get a code coverage report from a successful job even if the pipeline is unsuccessful. Potentially just retrieve the latest pipeline, rather than the latest successful pipeline. Or have the option to do so. This would be an option customers can enable when specifying the regex for their coverage.

Permissions and Security

No specific permissions indicated in the current documentation.

Documentation

Will update the test code coverage badge documentation.

What does success look like, and how can we measure that?

Users will be able to see their code coverage badge display correctly for a successful job regardless if the pipeline was successful.

Links / references

cc/ @brendan

Edited by drew stachon