Skip to content

Calculate xMAUs unions for Redis HLL metrics

What does this MR do?

This MR demonstrates proposed implementation for new feature (https://gitlab.com/gitlab-org/product-analytics/-/issues/421) that will allow for calculation unions over multiple tracked metrics in order to get more accurate data for GMAUs and SMAUs metrics aggregation.

Baseline idea is as follow:

  1. PM defines events combination in new YAML file lib/gitlab/usage_data_counters/aggregated_metrics.yml
  2. As the final step of usage ping generation, when all events are already calculated new method Gitlab::UsageDataCounters::HLLRedisCounter.combined_events_data is called
  3. .aggregated_metrics_data reads content of lib/gitlab/usage_data_counters/aggregated_metrics.yml and calculates number of unique elements in union of all listed metrics for given events combination, with usage of pfcount

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Change is hidden behind feature flag - [ ] Changelog entry - [ ] Documentation (if required)

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Mikołaj Wawrzyniak

Merge request reports