Skip to content

Generate spec mapping with crystalball using test coverage

Albert Salim requested to merge caalberts-dynamic-map-crystalball into master

Generate spec mapping with crystalball and upload the mapping to S3 with other test metadata.

This MR contains a few changes:

  1. Add of crystalball to rspec, using a custom CoverageLinesStrategy. A custom strategy is needed because of SimpleCov's use of Coverage.start(lines: true).
  2. Add script to parse crystalball data, which is a yaml containing a list of files covered by each rspec example, into a JSON mapping of file to a list of rspec files.
  3. Add script to compress JSON mapping to be stored in S3
  4. Modify CI update-tests-metadata to generate and upload JSON test mapping in scheduled pipelines.

part of #263421 (closed), #222369 (closed)

Spike Findings

  • does not work out of the box with SimpleCov as both uses Coverage. Throws error as shown in https://gitlab.com/gitlab-org/gitlab/-/jobs/787324441. This is due to SimpleCov using Coverage.start(lines: true) and Crystalball using Coverage.start. We'd need to customize Crystalball strategy with Coverage.start(lines: true)
    • This is solved by overriding the strategy and detector to use Coverage.start(lines: true)
Edited by Albert Salim

Merge request reports