## Why As we ramp up to become a more data driven company, we need a scalable process in place to manage our product analytics. As PMs/Ems we each need to own and manage our metrics, as subject matter experts, to enable the business to understand how our stages and groups are being adopted. This has become a tedious and error prone process, resulting in the Data, Product Intel, Sales, and CS teams unable to interpret or report on some metrics due to lack of documentation or definitions. This is a major step in standardizing our data structures and preparing for Project Compass, future expansion in metrics, and analysis. This is also one of the steps needed to complete one of [Products OKRs](FY22-Q1 OKR: EVP Product: Increase Stages per Organization from 1.53 to 1.56) for Q1 FY22. For PMs, this will make it much easier to perform self service analysis and sisense reports, and communicate with stakeholders, as every metric across product will be easily searchable, with clear definitions. ## What We currently have `1413` metrics in Usage Ping json payload, many of which are missing information, definitions, etc. The Product Intelligence and Data Teams, do not have the insight into every metric added to the system, and in order to move forward with a simple, easy to use process for event metrics, we need to audit, and ensure all current metrics are properly documented. On 2021-02-17 we introduced [Metrics Definition YAML](https://docs.gitlab.com/ee/development/usage_ping/metrics_dictionary.html) files as a single source of truth. Using these YAML files we automatically generate the [Metrics Dictionary](https://docs.gitlab.com/ee/development/usage_ping/dictionary.html). This allows us to deprecate the current spreadsheet, move to an automated process, and easily add and investigate metrics going forward. *Note:* With https://gitlab.com/groups/gitlab-org/-/epics/4174 we gather all informations in [Event Dictionary](https://docs.google.com/spreadsheets/d/1VzE8R72Px_Y_LlE3Z05LxUlG_dumWe3vl-HeUo70TPw/edit?usp=sharing), now is the moment to move the dictionary back into our docs. The spreadsheet is deprecated, all the metrics should be in code. ## How We have the issues created per group in order to review and update if is the case the related information. The Metrics Dictionary includes all product analytics metrics and events we collect at GitLab in **Usage Ping**. It includes the following details: | Field | Required | Additional information | |---------------------|----------|----------------------------------------------------------------| | `key_path` | yes | JSON key path for the metric, location in Usage Ping payload. | | `description` | yes | Clear description of the metric. Is it an event? is it a user specific metric? | | `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml) | | `product_stage` | no | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. | | `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. | | `product_category` | no | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the metric. | | `value_type` | yes | `string`; one of `string`, `number`, `boolean` | | `status` | yes | `string`; status of the metric, may be set to `data_available`, `implemented`| | | `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none` | | `data_source` | yes | `string`: may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `ruby`. | | `distribution` | yes | The [distribution](https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/#definitions) where the metric applies. | | `tier` | yes | The [tier]( https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/) where the metric applies. | | `milestone` | no | Optional; The milestone when the metric is introduced. | | `milestone_removed` | no | Optional; The milestone when the metric is removed. | | `introduced_by_url` | no | Optional; The URL to the Merge Request that introduced the metric. | ## When We're targeting 13.11 to complete this work. This allows enough time for PMs to specify the definitions during 13.10 and schedule the work with their teams for 13.11. If that due date can not be achieved for your group please reach @kokeefe and the Product Intelligence team to come up with an alternative plan. ### How is this related to Performance Indicators? Having a complete Event Dictionary of metrics / events makes defining the XMAU metrics much easier. Each metric in the Event Dictionary can potentially be used as an AMAU. AMAU's are the building blocks of our GMAU, SMAU, Section MAU, and MAU Performance Indicators. To better understand how these are all related, see [Structure of Product's Performance Indicators](https://about.gitlab.com/handbook/product/performance-indicators/#structure). ### Completing the Metrics Dictionary The goal of this issue is to review and update the YAML Metrics Definition for every group. We imported `1413` YAML files. Most of them are assigned to the group that responsible of them according to [Event Dictionary](https://docs.google.com/spreadsheets/d/1VzE8R72Px_Y_LlE3Z05LxUlG_dumWe3vl-HeUo70TPw/edit?usp=sharing) Note: There might be renames of the groups, or reassignments of PM please help with assigning to the right group and PM. Attached to this Epic there are the issues created per group. ### Next Steps for Groups [See Metrics dictionary docs](https://docs.gitlab.com/ee/development/usage_ping/metrics_dictionary.html) [Checkout the Dictionary generated](https://docs.gitlab.com/ee/development/usage_ping/dictionary.html) 1. Work with your Engineering team or the Product Intelligence team, the updates on YAML files and Dictionary require gdk installation. 2. Check each YAML metric definition in issue description. 3. Check that all the metrics are assigned to the correct `product_section`, `product_stage`, `product_group`, `product_category`. 4. Add `description` of what your metrics are tracking. Work with your Engineering team or the Product Intelligence team if you need help understanding this. 5. Check that the `tiers` are correct, work with your Engineering team or the Product Intelligence team if you need help understanding this. 6. Check that the `distribution` is correct, work with your Engineering team or the Product Intelligence team if you need help understanding this. If metric is available under `ce`, `ee`, or both. If metric is available only under `ee` then it should be under `ee/config/metrics` 7. Check that the `value_type` is correct, work with your Engineering team or the Product Intelligence team if you need help understanding this. 8. Check that the `time_frame` is correct, work with your Engineering team or the Product Intelligence team if you need help understanding this. 9. Determine if the metric is deprecated or not used in any Dashboard and open a follow-up issue to update the status of the metric. 10. Generate the dictionary, work with your Engineering team or the Product Intelligence team if you need help understanding this ``` bundle exec rake gitlab:usage_data:generate_metrics_dictionary ``` 11. Ensure the tests are passing , work with your Engineering team or the Product Intelligence team if you need help understanding this ``` bundle exec rspec spec/lib/gitlab/usage/ ``` ### Example of Metrics Update MR https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54539 For any questions or suggestions reach out [#g_product_intelligence](https://gitlab.slack.com/archives/CL3A7GFPF) or to Product Intelligence group `@gitlab-org/growth/product-intelligence` We know this seems tedious, and every team is buried in other priorities, but actioning on this now will enable us to run a smoother data availability cycle and metrics reporting through standardized metric schemas and automated dictionary generation.