Skip to content

Improve incomplete / wrong information for gathering of JaCoCo test coverage percentage

Problem to solve

This concerns the code coverage feature for java projects using the JaCoCo parser. Unless i have missed a third doc page or something else, the way they currently describe the process simply cannot work following the examples they provide (at least for maven).

I followed the relevant documents to set everything up, which are

  1. project/merge_requests/test_coverage_visualization.md
  2. pipelines/settings.md

The issue: The example given for maven in 1. won't print the test coverage percentage to the log file, so it will never be picked up when using the RegEx example for JaCoCo given in 2.
One can make it work by simply adding a cat target/site/jacoco/index.html to the end of the test stage, however that will (a) print lots of HTML garbage to the job's log (making it larger than necessary as well as harder to read and debug) and it will (b) only give an integer percentage, which for most projects i too vague and also doesn't match the expected result displayed in the feature screenshot on 2.

Proposal

  1. Improve the maven example gitlab-ci.yml by adding
    - awk -F, '{ instructions += $4 + $5; covered += $5 } END { print covered, "/", instructions, " instructions covered"; print "Total", 100*covered/instructions "%" }' target/site/jacoco/jacoco.csv 
    as last script step to the maven test stage example on page 1.
    This will make some coverage statistics appear in the job's log such as:
    36569 / 126018  instructions covered
    Total 29.0189%
    If anyone knows a command that's more brief, that'd be more than welcome of course 😉
  2. change the example RegEx for Java/JaCaCo projects on page 2. to Total.*?([\.0-9]*)%
    (using the current one would show 189% code coverage in the example above 😆)
  3. Add some text, linking from page 1. to page 2. and vice-versa.
    Currently users need to be lucky (or perseverant) enough discover them individually, although they seem closely related, and, as far as i currently understand, configuring the gitlab-ci.yml as described in 1. is a must-have in order to be able to enable the feature outlined in 2. Still, they are distinct features (1. = display coverage line by line, 2. = display total coverage in badges, merge requests and repository graph)
  4. Also link from page 2. to this one: https://docs.gitlab.com/ee/ci/yaml/#coverage
    Clarify if setting the regex in gitlab-ci.yml takes precendence over setting it in the frontend.

With those in place we'd have a much more pleasant experience for users trying to enable this often wanted feature in gitlab (took me roughly three weeks until the penny dropped for me).

Edited by Phil