Skip to content

CI/CD coverage is not able to capture coverage generated by pytest-cov

Problem to solve

When using the coverage feature on a CI/CD job it can't properly grab the coverage value generated by pytest-cov terminal report.

I've configured my job to capture the coverage using:

 coverage: '/TOTAL.*\d+%/'

Since the coverage report is generated like the following:

----------- coverage: platform linux, python 3.6.6-final-0 -----------
Name                      Stmts   Miss  Cover
---------------------------------------------
integrade/__init__.py         0      0   100%
integrade/api.py             86      0   100%
integrade/config.py          36      1    97%
integrade/exceptions.py       3      0   100%
integrade/utils.py           11      0   100%
tests/test_api.py           153      0   100%
tests/test_config.py         46      0   100%
tests/test_integrade.py       6      0   100%
tests/test_utils.py          12      0   100%
---------------------------------------------
TOTAL                       353      1    99%

By having that the current coverage number I am getting it 353 which is the first number matched on that line. It would be better if I could customize how to match the coverage number.

Further details

Proposal

By allowing further customization of how to match the coverage number it would enable the coverage to be more flexible and support more coverage reporting applications. Also it would be great if the coverage would be validated so it would not report more than 100% coverage since it does not make any sense.

What does success look like, and how can we measure that?

Having a job where pytest-cov is generating the coverage report and having the coverage feature matching the proper number is the way to measure success. Also maybe it should validate the matched number is higher than 100% and fail.

Links / references

Here is how pytest-cov generates reports http://pytest-cov.readthedocs.io/en/latest/reporting.html