GitLab Next

⭐️ Product Performance Indicator - Number of Commits in the Web IDE per Month

📖 Background and Overview

• Product Performance Indicators Presentation
• Deck from Product Performance Indicators Presentation

▶ Define your product feature's key benefit/value to customers

The Web IDE helps users make contributions to projects via changes to the project's code base. Most of these contributions currently end with a commit as an indication that the user was able to accomplish what they wanted to accomplish.

🥅 Define Product Performance Indicator that quantifies delivery of this "must have" value

Currently the most important metric for the success of the Web IDE is that a user is able to accomplish what they wanted to accomplish and makes a commit.

Number of commits per month is the NSM for the Web IDE.

• Once defined, please review with this checklist and iterate if needed

• Does this support sustainable revenue growth?
• Does this reflect customer getting value?
• Does this measure progress?

• Why do you pick this Product Performance Indicator for your feature/group/stage?

The Web IDE currently has a variety of outcomes, but most of these outcomes have Commits as part of that activity. Some of the supporting metrics include:

• Number of Merge Requests Created: This is an important measure of a possible outcome and something that is highly valuable to the Web IDE
• Live Preview Sessions: This is an important measure of a particular feature of the Web IDE, but doesn't convey overall value of the Web IDE.

Risks As the Web IDE feature set continues to expand, commits may not be an accurate reflection of value in the Web IDE. This may mean things like code review or other non-contributing workflows become important. At that time it will be important to shift the NSM to Web IDE sessions so that we're appropriately accounting for all types of activity.

More Details gitlab-org/create-stage#12656 (comment 310729626)

❗ What do you need from the Telemetry/Data team?

• The data is available but I need analytics to visualize/re-structure
• The data is not available and I need to implement tracking
• I don't know, need help to check whether this data exists or not

• Please provide more details

Currently this dashboard exists which is tracking these metrics. I think there's a couple important clarifications to make:

1. The data needs to be normalized with GitLab.com now reporting (March 2020)
2. Data needs to be confirmed that it's per month and not cumulative
3. Because views data may become the NSM in the future, it would be valuable to start visualizing it the same way now

• What is the data source for the proposed Product Performance Indicator? (Link, Explanation of how this data is captured, etc)

Usage ping - available in analytics.version_

• Who would be the best contact for the data source system? (Link or explanation)

🧩Answer these questions to provide more context

• Any alternative Product Performance Indicators you would consider, either because they are easier to get, or you are debating between that and the proposed NSM? - We may consider views in the future.
• Does your feature play a bigger role in initial purchase, retention, or expansion/upgrade? - retention/expansion
• Does your feature play a bigger role in SMB sales, Enterprise sales, or both? Unknown - it's not paid.
• What are some dimensions you would expect to slice/dice your Product Performance Indicator? For example, by plan tier, by free/paid, etc. - Plan Tier, Free/Paid, Per User
• Are there any GitLab product features that are "downstream" or "upstream" to your product feature? For example, customers have to use feature A to use your feature, or once customers use your feature, they are more likely to use another feature.
• Please attach a mock-up, drawing, or chart in another system that will think will allow you to make a data-driven decision for the propose metric. Link: ______
  • Please provide an example of how you would use the mock-up or chart to make the data-driven decision.

📈 Define Supporting KPI/metrics - [Optional]

• Live Preview Sessions

• Review Tab Opened

• Merge requests started

• Once defined, please review with this checklist and iterate if needed

• Are these metrics an input into Product Performance Indicator?
• By improving these, do you improve the Product Performance Indicator?