Skip to content

Harden Usage Ping | Redis Counters

Problem

Currently in Usage Ping there are ~400 usage ping counters with varying implementations, this causes stability issues and in some causes an uncaught error can cause the entire usage ping to fail.

In lib/gitlab/usage_data.rb

  1. Gitlab::UsageDataCounters::WikiPageCounter, => redis?
  2. Gitlab::UsageDataCounters::WebIdeCounter, => redis?
  3. Gitlab::UsageDataCounters::NoteCounter, => redis?
  4. Gitlab::UsageDataCounters::SnippetCounter, => redis?
  5. Gitlab::UsageDataCounters::SearchCounter, => redis?
  6. Gitlab::UsageDataCounters::CycleAnalyticsCounter, => redis?
  7. Gitlab::UsageDataCounters::ProductivityAnalyticsCounter, => redis?
  8. Gitlab::UsageDataCounters::SourceCodeCounter, => redis?
  9. Gitlab::UsageDataCounters::MergeRequestCounter => redis?

In ee/lib/ee.gitlab/usage_data/*

  1. Gitlab::UsageCounters::DesignsCounter => redis?
  2. Gitlab::UsageDataCounters::LicensesList => redis?
  3. Gitlab::UsageCounters::DependencyList. => redis?
  4. Gitlab::UsageCounters::PodLogs => redis?

Note - For Redis Counters: this includes anything that uses redis, this however may be a "russian doll" where usage pings calculations are temporarily stored in Redis / multiple usage pings. We will need to spend time tracing this.

Proposal

Rework redis counters so they are isolated within a redis count method which is isolated with rescue fail safes.

Result

A more robust usage ping without single points of failure

Next steps (if any)

Edited by Sid Reddy