Harden Usage Ping

Overview

Usage ping in it's current state is fragile. If another stage team adds a complex counter to usage ping, we need to ensure the entire usage ping does not break.

Current state:

• If it's a database timeout, usage ping will work but the specific counter will simply provide back a value of -1. We''re currently trying to optimize these queries, here are some examples which counters are currently timing out: gitlab-org/gitlab#205281 (comment 298541599)
• If it's an uncaught error, it currently will break and other data will be lost. For example avg_cycle_analytics was giving an uncaught error: gitlab-org/gitlab!26381 (merged). We've implemented a fix for that. MVC
• On the versions app side which receives the payload, if dirty data in the usage ping is passed, it currently is not cached and the usage ping is discarded.

Next steps:

Isolate usage ping for robustness

• 12.10 Consolidate all counters into four main counters with fail safes gitlab-org/gitlab#212403 (closed)
  • Isolate Alternative Counter gitlab-org/gitlab#212400 (closed)
  • Convert Counter: Any left over counters convert into batch counters gitlab-org/gitlab#208923 (closed)
  • Convert Counter: Jira Usage gitlab-org/gitlab#212398 (closed)
  • Disallow distinct count for regular batch count gitlab-org/gitlab!28518 (merged)
• 13.0 Expand add_usage_data to all 400 counters
  • Usage Data Counter Interface gitlab-org/gitlab#212403 (closed)
  • Isolate Redis Counters gitlab-org/gitlab#212402 (closed)
  • Save original usage ping payload before parsing https://gitlab.com/gitlab-services/version-gitlab-com/-/issues/281