[Experiment Rollout] invite_members_for_task - Invite members for task
What
Track the status of an experiment through to removal.
- Experiment key:
invite_members_for_task
- Framework:
gitlab_experiment
- Feature flag name:
invite_members_for_task
This is an experiment tracking issue for: https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/432
using the scoped experiment label.
As well as defining the experiment rollout and cleanup, this issue incorporates the relevant
Feature Flag Roll Out
steps.
Owners
- Team:
group::activation
- Most appropriate slack channel to reach out to:
#g_activation
- Best individual to reach out to: @alexbuijs
Expectations
What are we expecting to happen?
What might happen if this goes wrong?
What can we monitor to detect problems with this?
Tracked data
experiments
table
id | name |
---|---|
experiment_id | invite_members_for_task |
experiment_subjects
table (note: we do not distinguish between the 3 variants in the database)
experiment_id | namespace_id | variant |
---|---|---|
experiment_id | namespace_id |
0 (control) or 1 (candidate) |
onboarding_progresses
table
namespace_id | git_write_at (task 0) | repository_imported_at (task 0) | pipeline_created_at (task 1) | issue_created_at (task 2) |
---|---|---|---|---|
namespace_id |
NOT NULL (converted) or NULL (not converted) |
NOT NULL (converted) or NULL (not converted) |
NOT NULL (converted) or NULL (not converted) |
NOT NULL (converted) or NULL (not converted) |
Snowplow tracking
The following events are tracked anonymously through Snowplow:
activity | category | action | context | label | property | sent from | variants |
---|---|---|---|---|---|---|---|
Link clicked in the email | groups:show |
modal_opened_from_email |
n/a | n/a | gitlab_experiment json schema | frontend | candidate |
Link clicked in the email | groups:show |
submit |
selected_tasks_to_be_done |
A string with any combination of ci , code , and issues , separated by a comma |
gitlab_experiment json schema | frontend | candidate |
Staging Test
Experiment tracking log
Experiment Results
Roll Out Steps
-
Confirm that QA tests pass with the feature flag enabled (if you're unsure how, contact the relevant stable counterpart in the Quality department) -
Enable on staging ( /chatops run feature set invite_members_for_task 50 --actors --staging
) -
Test on staging -
Ensure that documentation has been updated -
Enable on GitLab.com for individual groups/projects listed above and verify behaviour ( /chatops run feature set --project=gitlab-org/gitlab feature_name true
) -
Coordinate a time to enable the flag with the SRE oncall and release managers - In
#production
mention@sre-oncall
and@release-managers
. Once an SRE on call and Release Manager on call confirm, you can proceed with the rollout
- In
-
Announce on the issue an estimated time this will be enabled on GitLab.com -
Enable on GitLab.com by running chatops command in #production
(/chatops run feature set invite_members_for_task 50 --actors
) -
Cross post chatops Slack command to #support_gitlab-com
(more guidance when this is necessary in the dev docs) and in your team channel -
Announce on the issue that the flag has been enabled -
Remove experiment code and feature flag and add changelog entry - a separate cleanup issue might be required -
After the flag removal is deployed, clean up the feature flag by running chatops command in #production
channel -
Assign to the product manager to update the knowledge base (if applicable)
Rollback Steps
-
This feature can be disabled by running the following Chatops command:
/chatops run feature delete invite_members_for_task
Edited by Alex Buijs