Skip to content

Execute related jests specs for MRs with backend changes

Plan

  • Run jest --findrelatedTests on MRs that contain backend changes
  • Incremental benefits:
    • shorter backend pipelines as only jest tests that depend on affected fixtures are run

Overview

flowchart LR
    subgraph frontend
    fe["FE code"]--jest --findRelatedTests-->jest
    end
    subgraph backend
    be["BE code"]--crystalball-->rspec
    end
    
    be--?-->fixtures
    fixtures--jest --findRelatedTests-->jest

The diagram above illustrates the test dependency between frontend and backend code and the respective tests. ? denotes the missing link needed in order to run related jest tests based on a backend change.

Some jest tests load the fixtures as dependency. In #339346 (closed), we have made it possible for jest to track the fixture dependency of the tests, so jest is able to run the relevant tests for each fixture file.

The next step from here is to identify which fixture file is affected by the backend change, so that when a backend file changes, we can feed jest --findRelatedTests with the fixture that was affected.

How it works?

A fixture rspec file generates a fixture file per example, based on the test description. For example, the following fixture rspec file

# spec/frontend/fixtures/pipelines.rb
  it 'pipelines/pipelines.json' do
    # ...
  end

  it "pipelines/test_report.json" do
    # ...
  end

generates the following test fixtures:

tmp/tests/frontend/fixtures-ee/pipelines/pipelines.json
tmp/tests/frontend/fixtures-ee/pipelines/test_report.json

The work in #339346 (closed) allows us to find the test files that uses this fixture:

$ jest --config jest.config.js --listTests --findRelatedTests tmp/tests/frontend/fixtures-ee/pipelines/pipelines.json tmp/tests/frontend/fixtures-ee/pipelines/test_report.json

ee/spec/frontend/pipelines/pipelines_table_spec.js
spec/frontend/pipelines/pipelines_spec.js
...

What is needed?

To make the link between ruby code and the fixture file, we need the following:

  1. A mapping from the source code to the fixture rspec file, for example:
{
  "app/controllers/projects/pipelines_controller.rb": ["spec/frontend/fixtures/pipelines.rb"]
}
  1. A mapping of a fixture rspec file to the fixture file itself, for example:
{
  "spec/frontend/fixtures/pipelines.rb": [
    "tmp/tests/frontend/fixtures-ee/pipelines/pipelines.json",
    "tmp/tests/frontend/fixtures-ee/pipelines/test_report.json"
  ]
}

With these mappings, we could derive the fixture files that might be affected, then pass them on to jest.

Given a change in app/controllers/projects/pipelines_controller.rb, spec/frontend/fixtures/pipelines.rb and therefore tmp/tests/frontend/fixtures-ee/pipelines/pipelines.json, tmp/tests/frontend/fixtures-ee/pipelines/test_report.json might also change. So we would need to run the following jest command:

jest --findRelatedTests tmp/tests/frontend/fixtures-ee/pipelines/pipelines.json tmp/tests/frontend/fixtures-ee/pipelines/test_report.json

Which would run the known jest tests that use the above fixtures:

ee/spec/frontend/pipelines/pipelines_table_spec.js
spec/frontend/pipelines/pipelines_spec.js
...

Implementation detail

To generate the fixture rspec mapping, we could use Crystalball to track the code executed by each fixture rspec file. This is similar to how we generate the mapping for other rspec files.

In addition, we need to map the fixture rspec file to the generated test fixture file. To do this, we need to hook into the rspec example execution and store the path of the generated file. We might be able to extract information from the helper for this purpose.

Combining the two, we could generate a map such as the following, which can be used to identify the fixtures that need to be passed to jest:

{
  "app/controllers/projects/pipelines_controller.rb": [
    "tmp/tests/frontend/fixtures-ee/pipelines/pipelines.json",
    "tmp/tests/frontend/fixtures-ee/pipelines/test_report.json"
  ]
}
Edited by Albert