Use test detection based on crystalball mapping in rspec jobs
What does this MR do?
This MR prepares jobs to detect relevant tests for an MR. This is part of approach outlined in #222369 (comment 457663853). This MR will be followed by !48741 (merged)
- Download test mapping artifact from latest
update-tests-metadata
master
job that produced crystalball mapping - Use test mapping with tff to detect relevant tests to the MR
- Add option rspec using
tooling/bin/parallel_rspec
with--filter
on matching tests, based on variableRSPEC_TEST_MAPPING_ENABLED
Test detection scope:
- We use GitLab ruby code coverage as a basis to determine the files for which we could detect tests for. Based on the coverage, the groups defined in simplecov would be covered: https://gitlab.com/gitlab-org/gitlab/blob/6128d8c312d20c5b70e773245457a00f6295a71d/spec/simplecov_env.rb#L58-75
-
spec
code is not tracked in coverage, so changes to code inspec
such asspec/factories
orspec/support
may not be detected. Need to verify these.-
verify that spec/factories
are covered: https://gitlab.com/gitlab-org/gitlab/-/jobs/879667810#L55 -
verify that spec/support
are covered: https://gitlab.com/gitlab-org/gitlab/-/jobs/879672410
-
Sample test pipelines:
- with change on
app/models/user.rb
: https://gitlab.com/gitlab-org/gitlab/-/pipelines/222103409 - 5309 test files
Part of #263407 (closed) #222369 (closed)
Pipeline Changes
This Merge Request contains changes to the pipeline configuration for the GitLab project.
Please consider the effect of the changes in this Merge Request on the following:
- Effects on different pipeline types
- No immediate change, as behaviour change on rspec is hidden behind flag
RSPEC_TESTS_MAPPING_ENABLED
- No immediate change, as behaviour change on rspec is hidden behind flag
- Effects on non-canonical projects (
gitlab-foss
,security
, etc)- No immediate change, as behaviour change on rspec is hidden behind flag
RSPEC_TESTS_MAPPING_ENABLED
- No immediate change, as behaviour change on rspec is hidden behind flag
- Effects on pipeline performance
- Additional API calls in retrieving test metadata to
- find master pipelines
- find job id with correct artifact
- download artifact
- Additional API calls in retrieving test metadata to
- Effects on fork pipelines
- No immediate change, as behaviour change on rspec is hidden behind flag
RSPEC_TESTS_MAPPING_ENABLED
- No immediate change, as behaviour change on rspec is hidden behind flag
Please consider communicating these changes to the broader team following the communication guideline for pipeline changes
Edited by Albert Salim