CI/CD coverage is not able to capture coverage generated by pytest-cov
Problem to solve
When using the coverage feature on a CI/CD job it can't properly grab the coverage value generated by pytest-cov terminal report.
I've configured my job to capture the coverage using:
coverage: '/TOTAL.*\d+%/'
Since the coverage report is generated like the following:
----------- coverage: platform linux, python 3.6.6-final-0 -----------
Name Stmts Miss Cover
---------------------------------------------
integrade/__init__.py 0 0 100%
integrade/api.py 86 0 100%
integrade/config.py 36 1 97%
integrade/exceptions.py 3 0 100%
integrade/utils.py 11 0 100%
tests/test_api.py 153 0 100%
tests/test_config.py 46 0 100%
tests/test_integrade.py 6 0 100%
tests/test_utils.py 12 0 100%
---------------------------------------------
TOTAL 353 1 99%
By having that the current coverage number I am getting it 353 which is the first number matched on that line. It would be better if I could customize how to match the coverage number.
Further details
Proposal
By allowing further customization of how to match the coverage number it would enable the coverage to be more flexible and support more coverage reporting applications. Also it would be great if the coverage would be validated so it would not report more than 100% coverage since it does not make any sense.
What does success look like, and how can we measure that?
Having a job where pytest-cov is generating the coverage report and having the coverage feature matching the proper number is the way to measure success. Also maybe it should validate the matched number is higher than 100% and fail.
Links / references
Here is how pytest-cov generates reports http://pytest-cov.readthedocs.io/en/latest/reporting.html