Manage:Compliance North Star Metric

Background and Overview

• North Star Metrics Presentation
• Deck from North Star Metrics Presentation

Summary

The key value that groupcompliance delivers to customers is saving the time and expense of managing the compliance of their GitLab groups and projects, which can number in the hundreds and thousands. It is difficult to quantify this value when the cost of that overhead lives outside of our scope. To measure with the data we have, or will have, available to us, we can focus on the actions that these customers take that are part of "managing compliance".

groupcompliance will measure multiple metrics to derive the current North Star Metric: Number of "compliance actions" taken by eligible customers.

North Star Metric

Number of "compliance actions" taken by eligible customers

This consists of:

• Hitting audit events API
• Setting admin push rules
• Number of MRs merged using admin push rules
• Number of MRs merged using approval settings
• Any search performed in GitLab SaaS audit events tables
• Use of protected branches (not including master)

• Why do you pick this north star metric for your feature/group/stage?

I've chosen multiple metrics as a means of doing validation on the long-term north star metric. The rationale here is selecting the actions that our compliance-minded customers take on a regular basis when managing compliance within GitLab.

Given the generally-passive nature of Compliance features, we need to rely on several signals rather than a single point of interest right now. Eventually, using this data and future feature implementations, we can narrow the focus of our north star metric and consider selecting one for each Compliance group category.

Next Steps

• Finalize the north star metric
• Work with the data team to ensure this data exists and/or can be collected
• Implement missing data collection
• Measure results and iterate on the metric definition