Experiment: Engineering - MVC add videos to continuous onboarding
Experiment Summary
We believe we can increase trial and feature adoption by including videos in the product experience. To verify this we will run an experiment where the control receives the current continuous onboarding experience and the candidate group has additional links to watch videos that related to specific features. We will measure the success of this experiment by monitoring the click rates on the video links and monitoring of the users that click the links to see if they adopt trials and the specific features at a higher rate than the control.
Design
Design will be completed in #351914 (closed)
Rollout strategy
50/50 AB test with enrollment at the namespace level at the time that the Continuous Onboarding page is accessed.
Inclusions and exclusions
n/a
Segmentation
n/a
Tracking Details
We should ensure that namespace, project, and pseudonymized user id are passed along with the click event in order to join to data sets for trial and feature adoption.
Questions we want to be able to answer:
- Do people engage with the videos?
- Do the free users that engage with the videos adopt trials at a higher rate than the control?
- Do the trial users that engage with the videos adopt the features at a higher rate than the control?
- Is there a signal about impact to paid conversion?
-
json schema used in
gitlab-experiment
tracking. - see taxonomy for a guide.
sequence | activity | category | action | label | context | property | value |
---|---|---|---|---|---|---|---|
load the continuous onboarding page | assignment | continuous_onboarding_videos |
assignment |
json schema | |||
2 | click on video link | projects:learn_gitlab:index |
click_video_link |
video name | json schema | Growth::Conversion::Experiment::LearnGitLab |