Analytics Instrumentation Planning and Prioritization FY26

Overview

The goal of this issue is to reassess our priorities and ensure we are focusing on the right objectives in the correct order.

The group's overarching aim is: enabling GitLab and its customers to make data-driven decisions using high-quality product usage data.

This would mean:

  • We are providing infrastructure and tools that allow teams to efficiently instrument with minimal support
  • Ensuring the infrastructure we provide cover all use cases at GitLab
  • Ensure high instrumentation coverage across features
  • Ensuring accuracy and reliability of the data we help collect
  • Ensuring compliance with legal and privacy policies
  • Ensure the data we collect can be seamlessly transformed into actionable insights, as data holds little value unless it drives meaningful outcomes. This could be through supporting data unification efforts or ensuring that our data is compatible with downstream systems or finding ways to directly report on the data we collect.

Prioritisation for FY26

While prioritising for FY26, following are considered

  1. Our overarching goals
  2. Key product investment theme for FY26, which is: **Standardised instrumentation tooling for all use cases** and Data Quality:Without more visibility, we cannot make as many data-driven decisions. And neither can our customers. We need to complete our data unification effort to make this easier.
  3. Technical roadmap

Projects and Prioritisation

RICE framework scoring details here

Project Reach Impact Confidence Effort Score Notes Status

Event data collection for SM and Dedicated instances

10 3 1 4 7.5

Reach, impact are marked in the highest

for reason detailed in our FY26 product

investment theme.

Confidence is marked high because

we already have a detailed roadmap.

Complete

Standardise product usage data tracking beyond the main Gitlab monolith [1,2]

10 3 0.8 2 12
Reasoning Marking reach and impact as high given the current focus on AI and potential growth in cloud connector features. The requirement is quite clear but I'm not sure we know what the specific solution should be so marking confidence as Medium. Effort will mostly revolve around communication with data team.

Track feature instrumentation coverage and build solutions to improve it

8 3 0.5
Reasoning Given the low instrumentation rates we are currently seeing, I'm marking reach and impact on the higher end. Confidence is low since we are not fully clear on what is stopping teams from instrumenting and what the specific solution should be. Effort is based on the assumption that we first need to link metrics/events to features and then build automations around it.

Improving ease of instrumentation

10 0.5 0.5 1 2.5
Reasoning Reach score of 10 because it impacts all our internal users. Impact score between Medium and massive(x2.5) because potential impact on better data-based decisions with GitLab. Confidence is low (50%) because the problems and solutions will become more apparent once we get more teams to attempt instrumenting features. Effort based on the assumption that it will mostly be many smaller individual changes.

Remove need to use deprecated event tracking

1.5 2 1 1
Reasoning Marking reach as small since I think we cover most significant use cases covered by the previous method. Impact is low since teams can always go back to using the old method and it's not a blocker. Confidence is high since we know what the specific use cases are. Effort is comparatively low if we assume we just add the functionality in the same way as before.

Customer health score metrics moved to internal events

8 3 0.5
Reasoning Marking reach and impact higher since these are critical metrics that we eventually want to surface to external customers. Marking confidence low since we are not sure of the solution. There are only 24 event-based customer health score metrics left that are not internal events. Database-based metrics can't be easily migrated.
Completely remove outdated redisHLL and snowplow options from codebase 3 2 0.8 12

Enable .com customers to access usage data

3 2 0.5 12
Reasoning Reach is significant since we see some demand from customers for this data. Impact is high since customers use this to track and encourage usage, potentially increasing revenue. Effort is assuming full access to collected usage data which would span across collected events and database-based metrics.

Improve service ping performance

Data quality: Better data quality monitoring and alerts

10 2 1 2

Data quality: Ensure compliance to policy

Enable self-managed customers to derive insights from event data

*Note: Product analytics initiatives is not on this list because of our ongoing commitment to support PA

Desired Outcome

  • Prioritised list of projects based on RICE framework

  • Have issue/epic placeholders for sub projects

  • Build 1 yr and mid-term roadmap based on the list and update handbook

Edited by Tanuja Jayarama Raju