Line number mismatch in GitLab test coverage visualization
Steps to reproduce
Check the test coverage in an MR, e.g. !3036 (98202680)
Expected Behaviour
Adjacent lines without branching should have the same coverage.
Actual Behaviour
Adjacent lines without branching does not have the same coverage and some lines does not have any marking at all:
- line 181 has 0 hits.
- line 182 doesn't have any hit indication at all.
- line 183 has 55 hits
OS Version
Ubuntu 22.10.
Graphviz Version
N/A, but the main branch is at 00344efd.
Additional info
The generated HTML pages by both lcov and gcov does not exhibit this problem.
From the artifacts of the ubuntu-22.10-cmake-ASan-test-including-ctest job of !3036 (merged):
From the lcov report:
The line numbers are actually 175-177 and correctly have the same coverage. Lines 181-183 are not the lines indicated by GitLab.
From the gcov report:
This shows the same as the lcov report.
Checking out the https://gitlab.com/smattr/graphviz/-/tree/smattr/gitlab-1950 branch and checking the line numbers with an external editor shows that the lcov and gcov line numbers are correct and the line numbers in the GitLab UI are wrong.
From https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html#limits:
A limit of 100 nodes for Cobertura format XML files applies. If your Cobertura report exceeds 100 nodes, there can be mismatches or no matches in the merge request diff view.
A single Cobertura XML file can be no more than 10MiB. For large projects, split the Cobertura XML into smaller files. See this issue for more details. When submitting many files, it can take a few minutes for coverage to show on a merge request.
The visualization only displays after the pipeline is complete. If the pipeline has a blocking manual job, the pipeline waits for the manual job before continuing and is not considered complete. The visualization cannot be displayed if the blocking manual job did not run.
This seems like a plausible but absurd explanation. The issue mentioned is gitlab-org/gitlab#328772.