Metric compliance_total_unique_counts mismatch

Summary

Following service ping total metrics are broken:

  1. redis_hll_counters.compliance.compliance_total_unique_counts_weekly
  2. redis_hll_counters.compliance.compliance_total_unique_counts_monthly

Listed metrics aggregate data from following events:

  1. g_compliance_dashboard
  2. g_compliance_audit_events
  3. i_compliance_audit_events
  4. i_compliance_credential_inventory
  5. a_compliance_audit_events_api

All but last one reports visitor_id, while a_compliance_audit_events_api reports on user_id. Due to that fact counting uniq values aggregate might report incorrect data, because different data pieces are counted.

Steps to reproduce

Example Project

What is the current bug behavior?

Metric is unable deduplicate aggregated attribute due to it coming from two different sources, which can lead to counting some data multiple times

What is the expected correct behavior?

Events included in aggregate reports on the same data attribute, either user_id or visitior_id

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

  1. Remove broken metric, since GitLab instances running on older resales would not apply fix once it is released, which would lead to skewed data
  2. (optionally) Unify events reporting and add new aggregate
Edited by Sebastian Rehm