New admin upgrade design/copy for issue weights locked feature
overview
This is the first iteration related to this parent epic related to adding more value to the paywall states.
Hypothesis
We believe that it currently can be difficult for free admin users to find enough context and value in the issue weight locked state to warrant proceeding further in learning more about the feature, upgrading or starting a trial.
Design
To test this hypothesis, we will present a cohort of free admin users with a new issue weight locked state that contains more information and a contextually relevant image.
Current user experience
New experience
Flow in design tab of this issue.
ICE score
Impact | Confidence | Ease | Score |
---|---|---|---|
6 | 7 | 8 | 7 |
Implementation
This will go live on 100% of free GitLab.com accounts. This experience will only target admin users that have the permissions to proceed with an upgrade. We are purposefully launching this as a baseline improvement as we do not have the capabilities to run this as an experiment.
Possible future test
In the future when we're able to run a/b experiments we could rerun this as an experiment with a percent of net new sign-ups to prove the impact. This is a decision we will make in the future based on the need for an impact assessment while balancing our future backlog of work.
Experience by cohort:
- Control - admin: The admin users in the control group should receive the standard experience in the UI today. Two options today
upgrade plan
anddon't show me this again
. - Experiment - admin: The admin users in the experiment group will see a
learn more
to the right of issue weights at which point it will expand and the design/copy will be preset (outlined in the design section below). In the experiment version, there will be three links "learn more about theBronze Plan
" which will link to customer.gitlab/plan. TheUpgrade Now
CTA will also link to customer.gitlab/plan. The final option ofDon't show me this again
will follow the pattern today in the control of hiding the feature.
Success Metrics
-
Primary success metric: 5% increase in the primary CTA
Upgrade now
click-through-rate -
Exposures needed to reach significance: TBD (section will be provided in future experiments when we have access to usage data)
-
Estimated duration: TBD (section will be provided in future experiments when we have access to usage data)
Tracking
To track this experiment we need to be able to do the following for each cohort:
- Control - admin: Track the number of admin users that view an issue then the number of admin users that select
upgrade plan
then the number of admin users that select any of the three optionsupgrade your plan
don't show me this again
or theX
to close the module. We should also track the free to paid namespace upgrade rate for the control group. - Control - namespace - track the total number of issues open and closed
- Experiment - admin: Track the number of admin users that view an issue then the number of users that select the
learn more
(specifically thelearn more
toggle to open the window) next to "Issue weights" then the number of admin users that select any of the three optionsupgrade your plan
don't show me this again
or thelearn more
(specifically thelearn more
toggle to close the window) to close the module. We should also track the free to paid namespace upgrade rate for the experiment group. - Experiment - namespace - track the total number of issues open and closed