DevOps Score (forward-looking guidance) - Design Exploration
Problem
We need to start addressing 2 key challenges enterprise GitLab customers face:
- GitLab ROI & Value - Backward-looking justification - Understanding and demonstrating the ROI of GitLab by understanding project, group & instance-level activity (see GitHub Enterprise).
- Transformation progress & benchmarking - Forward-looking guidance - Understanding progress of DevOps transformation and how their company compares to others in similar industries, markets, etc. Getting advice on how they can further their progress.
We plan to start by tackling 1, with 2 as the end goal.
Ultimately, we are helping our users understand and communicate how their DevOps effort metrics are impacting business value and product management (customer) value. To do this we must help them understand how to define, measure, and track business value from different points of view — and ensure that developers deliver meaningful value for the business and its customers.
Target users
- Director of DevOps / CIO - Buyer Personas
- Parker, Product Manager - User Persona
- Gabe, Engineering Director - User Persona
Target Stakeholders
Engineering teams and business leaders often have different views of business value. It’s important to measure and translate software performance measures into metrics that stakeholders understand. Therefore, it's also worth considering the type of stakeholders (and their relative interests) who our target users are reporting to:
- Business stakeholders want financial-related metrics (e.g. impact on revenue, value of abandoned orders & increase in web traffic)
- Product management want customer-related metrics (e.g. CSAT, NPS)
- DevOps teams want productivity and efficiency-related metrics (e.g. Features released, cycle time, quality & velocity)
Eventually, we want to help our users communicate these metrics in the context of business value measures (from [Gartner Report: Sync Developers with Business Needs(/uploads/4d8bf911b9d7d4ef87a5222e6d93a8e3/Sync_Developers_With_Busi.pdf)):
Research
See DevOps Score - Problem Validation #605
Jobs to be done (Hypothesis) [WIP]
For simplicity, we can start by focusing on customers who are looking to justify...
- Upgrading from CE >> EE, Premium >> ultimate etc
- Renewing current plan
# | Situation | Motivation | Outcomes |
---|---|---|---|
1 | When I am justifying the purchase/renewal of GitLab to my business stakeholders | I want to demonstrate how GitLab adoption is improving my DevOps teams' productivity | so that they can understand the ROI of using GitLab |
2 | When I am streamlining my teams' processes & tools | I want to know what people are using GitLab for (e.g. no. of MRs/commits/issues/etc ) | so that I can understand how GitLab is being adopted and whether it is being utilised effectively |
3 | When I am justifying the renewal of GitLab to my business stakeholders | I want to know what people are using GitLab for (e.g. no. of MRs/commits/issues/etc ) | so that I can understand how GitLab is being adopted and whether it is being utilised effectively |
Solution
Opportunities
- HMW... Create a better relationship between KPIs, OKRs & business value in GitLab?
- HMW... Collaborate with ~"devops::release" on the use of DORA metrics in #37139 (closed)
- HMW... Justify the value of tier upgrades to customers as they continue to use the product?
Metrics
We can consider metrics at few different levels ([see breakdown]):
- Instance-level
- Group-level
- Project-level
Design Considerations
- Inspire friendly competition between teams
- Feature adoption and onboarding
- Shareable format with stakeholders
- Different views for different stakeholder interests in value
Approach
Timescale: ~ 2/3 weeks
- Review current requests from customers around this topic
- Create research plan & discussion guide
- Explore & prototype potential solutions
- Recruit customers from CAB for research discussion
- Run research Problem/Solution validation
- Iterate on vision and create MVC for execution in ideally 12.8
Further information
- Customer notes on topic
- https://www.enterpriseready.io/github/reporting/
- Jeremy's whiteboarding video
- https://help.github.com/en/enterprise/2.16/admin/installation/activity-dashboard
- Gartner Report: Sync Developers with Business Needs
- GitLab ROI Calculator: Upgrade CE >> EE
- GitLan ROI Calculator: New customers