Experiment library: issue templates, labels + handbook page?
Let's keep a library of all the experiments and document them consistently. I call this 'The Experiment Library' but please suggest your ideas for the name too. At any point, we should be able to dig up results or status of an experiment and share it with others if needed. For this documentation to be easy to read for others, we need to define a consistent way of documenting it. Here are a few ideas.
Issue template
The main discussion about the issue template is located at #32 (closed)
Old description:
Test card | Learning card |
---|---|
![]() |
![]() |
Ideas for things we could include:
- hypothesis
- test
- metric
- criteria
- observation
- learnings and insights
- decisions and actions
- Checklist (to help with the steps required for consistent documenting)
These things could simply be titles in an issue template with some guidance for each section.
The issue template would also add certain labels by default (TBD which ones?).
Issue labels
With a good structure of labels it would be easier to keep track of our experiments, their status and results.
Possible issue labels:
- experiment::backlog (?)
- experiment::in progress (?)
- experiment::validated
- experiment::invalidated
- experiment::inconclusive
The last three are the most important, I'm not sure if it makes sense to differentiate between 'backlog' and 'in progress'.
Handbook page
With a setup like this, our issues would be the SSOT for the experiments, their status and their results. On top of that, I think we could benefit from documenting certain experiments and their results on our Growth Handbook page. This way, it would be easier for others to see what we have been up to and what the results were, without digging into our list of issues. We could certainly link to the list from this page for the most up-to-date information. Not every experiment should be documented here as well, just the major ones with the biggest learnings. We would always link to the experiment issue.
cc @gitlab-org/growth I'd love to get everyone's thoughts on this.