Skip to content

Unify redis metric aggregation definition

Background

Because Redismetric aggregation feature was created before merics yml definitions and this caused legacy duplication and distribution of information. Event can be aggregated into metric with 3 of following ways:

  1. Total metrics - already deprecated feature, yet still used by some older metrics
  2. Aggregated metrics - via yaml config files at https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics/aggregates and https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/metrics/aggregates
  3. Metric yaml definition - via option.events

This makes create confusing experience for developers, and makes it harder to utilize aggregation information for any downstream systems as discussed at #370462 (comment 1067324388)

Goal

Unify way of defining metrics aggregation, possibly remove all other methods and adjust metrics yaml definition files to be able to convey information about metrics aggregation including both redis and database sources as well as union and intersection operators (AND and OR)

Implementation

  1. Migrate remaining total metrics instrumentation to metrics yaml definitions, this work is related to #368996 (closed) and can be covered by it
  2. Add support for metrics aggreggation attributes to metrics yaml definitions
  3. Add existing aggregated metrics definitions to metrics yaml definitions
  4. Change agggregated metrics implementation to read from metrics yamls definitions https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage/metrics/aggregates.rb#L10
  5. Remove legacy aggregated metrics definitions
  6. Add information about counted attribute to metrics yaml definition
Previous implementation plan 1. Migrate remaining total metrics instrumentation to metrics yaml definitions, this work is related to #368996 (closed) and can be covered by it 2. Enhance metrics yaml definitions and aggregated metrics feature to use metrics yaml as source for aggregated metrics definition 1. ~~Add support for metrics aggregation attributes to metrics yaml definitions~~ moved to #373830 (closed) 2. Add existing aggregated metrics definitions to metrics yaml definitions 3. Change aggregated metrics implementation to read from metrics YAML definitions https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage/metrics/aggregates.rb#L10 4. Remove legacy aggregated metrics definitions 3. Add information about counted attribute to metrics yaml definition

Testing

Ensure related tests are passing

Edited by Carlo Catimbang