## Problem Statement For the go to market teams to be efficient and effective, we need to properly select, design, and implement the right product usage data metrics for customer retention and growth. **_The things that get measured are the things that get done._** ## Requirements **Phase 1:** 1. [x] Identify the specific areas of focus (use cases, general, consumption pricing, etc.) 1. [x] Identify teams and team leads for each area of focus 1. [x] Determine success criteria **Phase 2:** 1. [ ] Assign responsibilities, timelines, and expectations 1. [ ] Schedule metrics to be reviewed and instrumented or rejected 1. [ ] Instrument metrics 1. [ ] Communicate to Data Team, Product Intelligence, CS Ops, and CS that metrics have been instrumented (on each relevant use case issue is fine) 1. [ ] Begin monitoring and evaluate learnings! ### Overview Video - how to achieve **[Product Metrics to Instrument](https://youtu.be/BZAo3t3y0oo)** (useful as a high level overview on the project) <details> <summary> ### Phase 1: Outlining required metrics: Completed </summary> ### Areas of focus ##### Priority 1 1. [**User engagement**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/170) - activated, weekly/monthly activity, engagement 1. [**Use case: SCM**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/172) - has the customer adopted SCM 1. [**Use case: CI**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/182) - has the customer adopted CI? 1. [**Use cases: Security/DevSecOps**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/171) - has the customer adopted DevSecOps? ##### Priority 2 1. [**Use cases: CD**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/173) - Has the customer adopted CD? 1. [**Agile Planning**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/181) - is the customer using GitLab for agile planning/management? 1. [**General infrastructure:**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/175) configuration, settings, performance, etc. - how is the health and scalability of the customer's infrastructure? ##### Priority 3 1. [**Use cases: GitOps**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/183) - Has the customer adopted GitOps? 1. Prioritized by Support and Security ##### Priority 4 1. [**Consumption pricing:**](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/174) storage and CI minutes - how is the customer's health of consumption purchases? </details> <details> <Summary> ### Phase 2: Scheduling and Instrumenting Metrics: Not Complete </Summary> 1. [ ] [User Engagement](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/474) - @amandarueda and @poffey21 1. [ ] [SCM Use Case](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/475) - @sarahwaldner and @k33g 1. [ ] [CI Use Case](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/473) - @jreporter and @jwoods06 1. [ ] [DevSecOps Use Case](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/476) - @hbenson and @mrleutz 1. [ ] [CD Use Case](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/477) - @kbychu and @jrreid 1. [ ] [Agile Use Case](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/478) - @mushakov and @kevinchasse 1. [ ] [General Infrastructure](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/480) - @joshlambert and @DarwinJS 1. [x] [GitOps Use Case](https://gitlab.com/gitlab-com/sales-team/field-operations/customer-success-operations/-/issues/479) - @nagyv-gitlab and @danielmarquard </details> ### Resources 1. [Metrics to Instrument](https://docs.google.com/spreadsheets/d/1lhzy2A-35gkPb0E5YXEj1a2mv88mB1bkCZV_FSIHmxs/edit#gid=0) spreadsheet to detail all required metrics 1. [Priority list of existing metrics ingested or to ingest into Gainsight](https://docs.google.com/spreadsheets/d/1ZR7duYmjQ8x86iAJ1dCix88GTtPlOyNwiMgeG_85NiA/edit#gid=0) 1. [Metrics Dictionary](https://docs.gitlab.com/ee/development/usage_ping/dictionary.html) 1. [Customer Success Product Usage Requirements](https://docs.google.com/document/d/14vYIyo-L_nAmmpsB2zswdAKOE_h00oirDJOGCX9BU8s/edit) ### Suggested metric formats 1. **User-based metrics** # of users who did [abc] 1. Last 28 days 1. Last 7 days 1. Example: "# of users who completed a merge request in the last 28 days" 1. **Event-based metrics** # of [actions performed] 1. **Total counts** # of [events/attributes/users] 1. Example: "# of runners" or "# of auto_devops_enabled projects" 1. **TRUE/FALSE** Whether an attribute is true or false 1. Example: "Whether shared runners are enabled or not" 1. **Power user metrics** # of users who did this action, and this action, and that action ## Success Criteria 1. Fulfill requirements for [Early Warning System](https://gitlab.com/groups/gitlab-com/sales-team/field-operations/-/epics/25) through articulating and measuring a customer's adoption by breadth (user engagement) and depth (feature adoption) 1. Ability to build trend analyses (monthly, weekly, annually, etc.) to track trends such as growth or reductions 1. Ability to analyze feature usage, user adoption, and renewal forecasting 1. Does **not** require TAM, SA, SAL, AE, or SDR input or involvement to confirm adoption status (i.e., works for small and large accounts in an automated fashion) 1. Adoption per customer and segment is clear and consumable by Product, CS(TAM,SA, and PS), Sales, Marketing, and others ### Areas of focus teams - **User engagement** - activated, weekly/monthly activity, engagement - SAs: @poffey21 - TAMs:@mrleutz - **Use cases: SCM**- has the customer adopted SCM? CI? - SAs: @svij - TAMs: @k33g - **Use cases: CI**- has the customer adopted SCM? CI? - SAs:@edmondchan - TAMs:@jwoods06 - **Use cases: Security/DevSecOps**- has the customer adopted DevSecOps? - SAs: @drivey1 @Joemckairnes - TAMs: @mrleutz @r.leite - **Use cases: CD** - Has the customer adopted CD? - SAs: - TAMs: @jrreid - **Use cases: GitOps** - Has the customer adopted GitOps? - SAs: @danielmarquard - TAMs: @jrreid - **Use cases: Agile Planning** - Has the customer adopted Agile Planning? - SAs: @jvaleriano @nsharma2 @vehernandez - TAMs: @kevinchasse @manuel.kraft - **General infrastructure:** - configuration, settings, performance, etc. - how is the health and scalability of the customer's infrastructure? 1. Prioritized by Support and Security - SAs:@DarwinJS - TAMs: @yofeldstein - **Consumption pricing:** -storage and CI minutes - how is the customer's health of consumption purchases? - SAs: - TAMs: