[Experiment Tracking] jobs_empty_state - Jobs empty state
Overview
This is an experiment tracking issue for: gitlab-org&4888 (closed) using the scoped experiment label.
As well as defining the experiment rollout and cleanup, this issue incorporates the relevant
Feature Flag Roll Out
steps.
Owners
- Team:
group::activation
- Most appropriate slack channel to reach out to:
#g_activation
- Best individual to reach out to: Alex Buijs
Expectations
Tracking events:
Category | Action | Label | Group |
---|---|---|---|
Growth::Activation::Experiment::JobsEmptyState |
click_button |
experimentation_subject_id |
control_group or experimental_group
|
What are we expecting to happen?
More events for experimental_group
than for control_group
.
What might happen if this goes wrong?
What can we monitor to detect problems with this?
Staging Test
To force this experiment on staging use ?force_experiment=jobs_empty_state
See https://staging.gitlab.com/alexbuijs/alex-test2/-/jobs?force_experiment=jobs_empty_state
Rollout plan:
- Runtime: 30 days or until we reach statistical significance
- We will roll this out behind a feature flag and expose this to 50% of users to start then ramp it up from there.
- feature flag based on experiment key
<experiment-key>
(seeexperimentation.rb
in GitLab, append '_experiment_percentage')
/chatops run feature set jobs_empty_state_experiment_percentage 50
Roll Out Steps
-
Enable on staging -
Test on staging -
Ensure that documentation has been updated -
Enable on GitLab.com for individual groups/projects listed above and verify behavior. -
Announce on the issue an estimated time this will be enabled on GitLab.com -
Enable on GitLab.com by running chatops command in #production
-
Cross post chatops slack command to #support_gitlab-com
(more guidance when this is necessary in the dev docs) and in your team channel -
Announce on the issue that the flag has been enabled -
Remove feature flag and add changelog entry -
After the flag removal is deployed, clean up the feature flag by running chatops command in #production
channel
Results:
Edited by Alex Buijs