[PM] "What's New" Notification in GitLab
Experiment summary
We believe that by surfacing a notification icon/interaction within gitlab which highlights our recent product enhancements (and perhaps future ones as well), we'll be able to drive additional awareness and engagement with new features, leading to more engaged accounts and reduced churn.
To verify that, we will 50/50 split test an interaction where we alert customers to the highlight features of a recent release. The messaging will be able to drive customers to a blog URL or a URL within the gitlab application (configurable) - NOTE: We may just want to drive to the release page on the blog for an initial MVC.
And we’ll measure the impact on engagement with the highlight features for the test vs control groups. We expect to see higher engagement and usage for the test group.
Hypothesis
GitLab customers and system administrators are unaware of all of the value and enhancements that GitLab is releasing each month. By highlighting them more prominently in the application, we can drive deeper engagement, improve retention and stickiness by increasing usage across the product, and increased upgrades to the newest releases.
Business problem
As a business, we want to ensure that people are using and engaging with the product enhancements that we are creating. Since we are releasing so much each month, it's hard for customers to keep up and it requires them proactively looking at our blog.
Experiment design & implementation
Create a UI interaction to alert customers to new features. We could add an icon to the top-right, or more subtly add an additional option to the user or question menu.
For an MVC, this can just be a "What's New" menu option that drives to the release page. For future iterations, we should consider the ability to include a number (like on issues and MRs) to catch the users' attention. Upon viewing/interacting with the alert then the number would clear. In order to support this we would need:
- an admin area to create new messages
- a standard structure for the messages (Character limits, URL that it points to, headline, body, targeting)
- A way to schedule the messages to display
- A way to edit or delete messages
- targeting - version, user role, billing package, .com vs. Saas
- Ability for admins to turn it off
Open Questions / Items
Notes from 1/13/20 Sync meeting:
-
Does the drawer become a layer on top of the other drawers there? (ex: lose access to right-side bar) @timnoah -
How does this work for self-managed folks that don’t update? - [ ]Ideally we use this as a method to make people excited to upgrade. We discussed knowing the milestone that an item is related to and presenting an "upgrade" CTA for instances that aren't on that version.
-
How do we handle air-gapped customers? -
Need to gracefully handle the error. Perhaps we can hide the icon entirely in the case where we aren't able to connect to our side.
-
-
Do users understand the version they’re on? How can we convey to them that these things aren’t yet available? -
We discussed knowing the milestone that an item is related too and presenting an "upgrade" CTA for instances that aren't on that version.
-
-
How can we show a CTA to upgrade? -
Bake CTAs/alternate CTAs into the approach
-
-
Do we collect 2 URLs? -
Link if application link is available -
Link for marketing/documentation
-
-
What types of targeting do we need? -
* .com vs. self-managed -
* subscription/license tier -
* paid vs. free vs. trial -
* admin vs. else
-
-
Phil: Interaction is different than the others - may be a little odd (dropdowns) @timnoah -
Phil: Star icon is used in other areas, maybe a present or bell? @timnoah -
Phil: Could we use the “Next” real estate that’s in .com? @timnoah -
Jay: Because it’s a top nav change - do we need to go through some committee or approval process? -
Tim: No team owns the navigation right now. For wider team/company, we should share as much as possible early and often. Tim will share in Eng weekly and ping across stage groups. Usually we share prototype videos with the wider team. -
Jay: Self-managed may need an API. -
MVC likely .com specific to start -
Complete work in What's New Epic: gitlab-org&3556
ICE score
Impact | Confidence | Ease | Score |
---|---|---|---|
8 | 8 | 4 | 6.67 |
Next Steps
-
Create canvas https://docs.google.com/document/d/1QsrsfwgVWtu9rjmgxozmnFlTnSBvh3mNNGgLgVZl_xs/edit?usp=sharing -
Solution Validation interviews
Known assumptions
Results, lessons learned, next steps
Checklist
-
Fill in the experiment summary and write more about the details of the experiment in the rest of the issue description. Some of these may be filled in through time (the "Result, learnings, next steps" section for example) but at least the experiment summary should be filled in right from the start. -
Add the label of the Growth subgroup that will work on this experiment. -
Mention the Product Manager, Engineering Manager, and at least one Product Designer from the group that owns the part of the product that the experiment will affect. -
Fill in the values in the ICE score table ping other team members for the values you aren’t confident about (i.e. engineering should almost always fill out the ease section). Add the ICE Score Needed label to indicate that the score is incomplete. -
Replace the ICE Score Needed with an ICE low/medium/high score label once all values in the ICE table have been added. -
Mention the [at]gitlab-core-team team and ask for their feedback.