Upgrade undercover to 0.9.0 and compare commits in CI
What does this MR do and why?
scripts/undercoverage (the undercoverage check, which runs in rspec:coverage since !265673 (merged)) takes from about 10s to several minutes in CI, depending on the merge request. In 30 rspec:undercoverage jobs from tier 3 MR pipelines on 2026-10-26 and 2026-10-27, it took a median of 12s, but 27–113s in 6 of them.
undercover 0.8.5 had two problems:
- It diffed the compare base against the index and the working tree. That reads every file in the checkout (about 115,000), as the
git diffin !262134 (merged) did: jobs restore the checkout from theclone-gitlab-repoartifact, so the file metadata in the index doesn't match the files on disk. - It looked up each file's coverage with a linear search over the about 25,000 files in the report, normalizing the path again for each of them, for every method and block of each changed file. Changes to big files took minutes.
undercover 0.9.0 looks coverage up directly, matches changed lines to code in linear time (#274), and adds --commits-only (#273), which compares the commits only. This MR:
- Upgrades undercover from 0.8.5 to 0.9.0. Its new dependencies (
prism,erubi) are already in the bundle, andimagenis no longer needed. - Passes
--commits-onlyin CI, where the checkout is the commit without local changes. Locally, undercover still checks uncommitted changes too. - Passes
--no-hint, to keep 0.9.0's new hint about UndercoverCI out of the output. - Removes the
Result#pretty_printpatch. It made undercover show branch coverage on lines without line coverage, which undercover does itself since 0.8.4 (#249). Thecompare_base_objpatch for shallow clones stays. - Stops printing
git diff --statbefore the check. It only printed the diff, and for the same reason as the first problem above it took 40s in !265673 (merged)'s pipeline.
When an MR only changes files that undercover excludes (for example specs, or .gitlab/ci), 0.9.0 prints WARNING: nothing was checked - all N candidate file(s) were skipped by --include-files/--exclude-files and passes, as 0.8.5 did without the warning.
Results
In job 17528116872 of this MR's tier 3 pipeline, a temporary commit ran this branch's scripts/undercoverage (0.9.0) and master's (0.8.5, installed next to the bundle) against the same coverage report, for three compare bases:
| Compare base | Changed .rb files |
0.9.0 | 0.8.5 | 0.9.0 again | Output |
|---|---|---|---|---|---|
| This MR's target branch commit | 0 | 2.0s | 10.2s | 2.0s | identical, exit code 0 |
| 3 commits before it on master | 13 | 2.1s | 95.9s | 2.1s | identical, exit code 0 |
| 10 commits before it on master | 42 | 2.2s | 183.3s | 2.4s | identical, exit code 1 (one method flagged in ee/app/services/concerns/secrets_management/secrets/read_metadata_service_helpers.rb) |
The temporary commit is removed from this branch.
With !265542 (merged) and !265673 (merged) merged, this MR's tier 3 pipeline 3005882647 compares with the medians of 113 tier 3 MR pipelines from 2026-10-26 and 2026-10-27, before any of these changes:
| Before (median) | !265673's pipeline | This MR (job 17589845884) | |
|---|---|---|---|
git diff --stat |
7s | 40s | removed |
undercover, with bundle exec |
12s | 15s | 4.3s |
| Last RSpec job to the end of the coverage tail | 371s | 328s | 276s |
That's about 95s off the tail, from one pipeline.
How to set up and validate locally
- Download the
coverage/lcov/gitlab.lcovartifact of a recentrspec:coveragejob intocoverage/lcov/gitlab.lcov, and also ascoverage/lcov/<name of your checkout directory>.lcov.touchboth so they're newer than your changes. - Commit a change that adds a method without specs, for example in
lib/gitlab/access.rb. - Run
time CI=true bundle exec scripts/undercoverage <commit before your change>on this branch, and on master. - Both report the new method and exit with 1. On this branch it takes a few seconds.
- Without
CI, an uncommitted change is reported too.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.