Skip to content

Implement Additional Assignment requirements for `tier_badge` experiment

Summary

We want to include additional assignment requirements for the tier_badge experiment to ensure we are being intentional in the audience included into the experiment. This ensures a good user experience + clean data.

Current assignment state:

project overview page loads

&6901 (comment 1362604244)

We include all namespaces that are not paid(a valid trial is considered paid). There are no exclusions other than that.

https://gitlab.com/gitlab-data/product-analytics/-/issues/974#note_1359213679

New assignment requirements:

Include:

Note: We updated the inclusion requirements for this experiment to include both new + existing free users &6901 (comment 1467053750)

We are only excluding free namespaces that have trialed before.

OLD:

  • "Newly signed up namespaces on Free that have not yet trialed" (rationale can be found here)
    • Check at day 14 if they're:
      • Still on the free tier
      • Have had no indication of starting a trial OR converting to paid
  • "They see the experiment experience"
    • Project overview page loads (existing as noted above^)

Exclude:

  • Free namespaces that have started a trial within 14 days of signup
  • Free namespaces that have converted to paid within 14 days of signup
  • Free namespaces that never view the project overview page

Note: To determine "sign up" we are using group creation date as discussed below

If relevant to the new ask, @dstull @rliu-int @jmontal started looking into this here

Edited by Paige Cordero (on Parental Leave)