Skip to content

Measure continuous onboarding experiment

What does this MR do?

Measure continuous onboarding experiment

Related to #322025

How to test locally

  • Enable one of the feature flags: Feature.enable_percentage_of_time(:learn_gitlab_b_experiment_percentage, 100) # or the a variant
  • Signup with a new user
  • Go to the "Learn Gitlab" or any other project
  • Click on "Learn Gitlab"

Troubleshooting

Since we require the "learn-gitlab" project to be available, which gets created in a background-job, it could be that the project didn't get created. This is because background-jobs are sometimes stuck in development. What to do then:

  • gdk stop
  • ps aux | grep runsv
    • If there is any other process beside grep, you need to run kill -9 <pid>
  • gdk start again and go through the sign up flow again.

Events

1. Snowplow Event: Page inits for "learn gitlab continuous onboarding page"

Page URL : http://localhost:3000/MYNAMESPACE/MYPROJECT
Event Category : projects:learn_gitlab:index
Event Action : page_init
Event Label : learn_gitlab
Event Property : Growth::Activation::Experiment::LearnGitLabB

2.Snowplow Event: For clicks on sidebar nav menu item which is triggered in addition to the first event above.

Page URL : http://localhost:3000/MYNAMESPACE/MYPROJECT
Event Category : projects:learn_gitlab:index
Event Action : click_menu
Event Label : learn_gitlab
Event Property : Growth::Conversion::Experiment::LearnGitLabA  #=> Alper: Growth::Activation::Experiment::LearnGitLabB

3. Snowplow Event: For clicks on A experiment card items

Page URL : http://localhost:3000/MYNAMESPACE/MYPROJECT/-/learn_gitlab
Event Category : projects:learn_gitlab:index
Event Action : click_link
Event Label : Create or import a repository
Event Property : Growth::Conversion::Experiment::LearnGitLabA

4. Snowplow Event: For clicks on B experiment card items

Page URL : http://localhost:3000/MYNAMESPACE/MYPROJECT/-/learn_gitlab
Event Category : projects:learn_gitlab:index
Event Action : click_link
Event Label : Invite your colleagues
Event Property : Growth::Activation::Experiment::LearnGitLabB

5. Experiment_users

select * from experiment_users order by id desc limit 1;
id            | 357
experiment_id | 14
user_id       | 364
group_type    | 1
created_at    | 2021-04-26 20:43:57.966548+03
updated_at    | 2021-04-26 20:44:06.315143+03
converted_at  | 2021-04-26 20:44:01.673355+03
context       | { "project_id": 225, "namespace_id": 549, "in_a_experiment_group": false, "in_b_experiment_group": true, "learn_gitlab_project_id": 226}

The context provides us details on the current user

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Alper Akgun

Merge request reports