Experiment Implementation: Update 'grouping' of tasks presented in Learn GitLab
Experiment Summary
Design
Upon completion of trial registration, the user will see either the Control or Candidate experience
Control vs Candidate Experience
Control (Existing Grouping) | Candidate (New Grouping) |
---|---|
- Tasks and their actions remain the same
- No changes to event tracking
Rollout strategy
This experiment will be rolled out 50/50
Target Population
This experiment is explicitly "targeting" Trial Signups. We have a hypothesis that free and trial signups may have different engagement and thus want to control equal cohorting into control and candidate to best analyze results.
Assuming the experiment is successful we will roll out this change to everyone on SaaS.
We also want to ensure we are only targeting "new" signups. A user who has been exposed to the existing Learn GitLab grouping should not be included in this experiment (Discussed with engineering in
Tracking Details
Tracking
We want to introduce tracking that helps us understand if the 'new grouping' leads to a change in
- Clicks on Learn GitLab tasks
-
json schema used in
gitlab-experiment
tracking. - see event schema for a guide.
sequence | activity | category | action | label |
---|---|---|---|---|
Learn GitLab task is clicked | label should be specific to each task clicked on |