Skip to content

Add an issue template for experiments

Matej Latin requested to merge matej-experiment-template into master

I went ahead and took the first stab at this. Very much inspired by @s_awezec's proposal in #32 (closed), but also added a section for the experiment summary and a checklist to guide us so that we'll be consistent in tracking the progress of experiments. Some of the things in the checklist are still TODOs (experiment status labels, for example, which are still being discussed in gitlab-org/growth/discussion#1).

The idea behind the Experiment summary is that we have a short summary of what we're doing with the experiment at the very top of the description. This way it's very easy and quick to understand what the experiment is about. It should also provide a way for creating issues for experiments quickly—filling in the summary only at first—and adding more details later on.

Things we should resolve in a follow-up MR to update the template:

  • experiment status labels
  • how do we use milestones?
  • how do we use weights?

These are being discussed in gitlab-org/growth/discussion#1 and gitlab-org/growth/discussion#4

Checklist items to be considered in a follow-up MR:

* [ ] Indicate the status of the experiment by adding an [experiment status label](TBD-handbook). Remember to update the label as the experiment progresses.
* [ ] Once the experiment was completed, change the experiment status label to indicate whether it was validated, invalidated or inconclusive. Add the ~experiment::validated::clean_up_for_production label if further work needs to be done in order to get it ready for production. Close this issue.

Closes #32 (closed)

Edited by Matej Latin

Merge request reports