Skip to content

Ensure all variants for an experiment are covered by tests

Serhii Yarynovskyi requested to merge 407233-rubocop-rule-to-ensure-experiments-have-tests into master

What does this MR do and why?

New rubocop rule to ensure all experiment paths (control/candidate/multivariant/etc) are tested in code. This rule finds experiment classes or blocks in the code:

class MyCoolExperiment < ApplicationExperiment
  control
  candidate
  ...
end

experiment(:my_cool_experiment) do |e|
  e.candidate { ... }
  e.run
end

The rule validates if corresponding test files contain uncommented stub_experiments(... then experiment name and all the variants. You still can fool the rule if stub some dummy experiment and just use real experiment name/variant in comments somewhere. However this rule should be treated as a reminder to cover experiment branches with tests.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #407233 (closed)

Edited by Serhii Yarynovskyi

Merge request reports