Experiment Tracking: Default terms opt in
Overview
This is an experiment tracking issue for: #161 (closed) using the scoped experiment labels.
As well as defining the experiment rollout and cleanup, this issue incorporates the relevant
Feature Flag Roll Out
steps.
Owners
- Team:
group::acquisition
- Most appropriate slack channel to reach out to:
#g_acquisition
- Best individual to reach out to: PM @jstava / Engineering @nicolasdular
Expectations
What are we expecting to happen?
Increase sign up conversion
What might happen if this goes wrong?
Successful signups could decrease
What can we monitor to detect problems with this?
Form completion counts for this page vs the control
Staging Test
Can't be tested on staging because sign up is disabled
Rollout plan:
See #161 (closed)
- Runtime: 30 days or until we reach statistical significance
- We will roll this out behind a feature flag and expose this to 50% of users to start then ramp it up from there.
- feature flag
FEATURE_FLAG
(seeexperimentation.rb
in gitlab, append '_experiment_percentage')
/chatops run feature set terms_opt_in_experiment_percentage 50
Roll Out Steps
-
Enable on GitLab.com and verify behavior. -
Announce on the issue an estimated time this will be enabled on GitLab.com -
Enable on GitLab.com by running chatops command in #production
-
Cross post chatops slack command to #support_gitlab-com
(more guidance when this is necessary in the dev docs) and in your team channel -
Announce on the issue that the flag has been enabled -
Remove feature flag and add changelog entry -
After the flag removal is deployed, clean up the feature flag by running chatops command in #production
channel
Results:
Edited by Phil Calder