Experiment Tracking: New Create Project UI

Overview

This is an experiment tracking issue for: #210599 (closed) using the scoped experiment labels

!32606 (merged)

new_create_project_ui: {
  tracking_category: 'Manage::Import::Experiment::NewCreateProjectUi'
}

Staging Test

To force this experiment on staging use ?force_experiment=new_create_project_ui

Rollout plan:

• Start by exposing this to 50% of applicable users.
• Runtime: 30 days or until we reach statistical significance
• Eventually either rollout to 100% or disable the feature flag.

/chatops run feature set new_create_project_ui 50

Action dates

• 2020-06-xx - Enabled feature flag and exposed to 50% of users on gitlab.com
• 2020-xx-xx -

Beta groups/projects

N/A

Tracking

We are collecting metrics on the tab/panel clicks to be able to measure user interaction from both variants, to be able to accurately measure whether there's an improvement in engagement or not.

Activity being tracked	track_event	track_label	track_property
Selecting "Blank project" tab in old ui	click_tab	blank_project	control_group
Clicking "Blank project" card in new ui	click_tab	blank_project	experimental_group
Selecting "Create from template" tab in old ui	create_from_template	blank_project	control_group
Clicking "Create from template" card in new ui	create_from_template	blank_project	experimental_group
Selecting "Import project" tab in old ui	click_tab	import_project	control_group
Clicking "Import project" card in new ui	click_tab	import_project	experimental_group
Selecting "CI/CD for external repo" tab in old ui	click_tab	cicd_for_external_repo	control_group
Clicking "CI/CD for external repo" card in new ui	click_tab	cicd_for_external_repo	experimental_group

Tracking category: "Manage::Import::Experiment::NewCreateProjectUi"

Dashboard

Here is the Sisense dashboard: https://app.periscopedata.com/app/gitlab/688341/Manage---Import---New-Create-Project-UI-Experiment

Roll Out Steps

• Enable on staging
• Test on staging
• [-] Enable on GitLab.com for individual groups/projects listed above and verify behavior
• Coordinate a time to enable the flag with #production and #g_delivery on slack.
• Announce on the issue an estimated time this will be enabled on GitLab.com
• Enable on GitLab.com by running chatops command in #production
• Cross post chatops slack command to #support_gitlab-com (more guidance when this is necessary in the dev docs) and in your team channel
• Announce on the issue that the flag has been enabled
• Remove feature flag and add changelog entry
• Ensure that documentation has been updated
• After the flag removal is deployed, clean up the feature flag by running chatops command in #production channel

Results:

To be summarized in our Sisense dashboard: https://app.periscopedata.com/app/gitlab/688341/Manage---Import---New-Create-Project-UI-Experiment

And then eventually updated back here.

Experiment Clean Up

Once this experiment is complete, if we decide to proceed with new UI (expected outcome) #267488 (closed) should be planned and executed

If we decide not to proceed with new UI we should close #267488 (closed) and proceed with new follow-up issue