Replace Redis Counters with Postgres Counters

Problem

Redis is a non-persistent data store and is currently used in Usage Ping for incrementing counts. If an instance is ever reset these counts are also reset. I believe we should eventually replace Redis counters with something that is more persistent such as a Postgres counter or tracking via Snowplow events.

An example of where this happened: https://gitlab.slack.com/archives/C8D1LGC23/p1586297573122200?thread_ts=1586271630.104500&cid=C8D1LGC23

Proposal

Replace all Usage Ping Redis counters with Postgres counters.

By moving these counters into Postgres, we will also be able to use GitLab.com Postgres imports to dashboard out all Usage Ping counters. Currently we cannot dashboard out Redis counters as that data is not imported into our data warehouse.

Result

More accurate usage ping counters

Next Steps

Investigate if this is a viable solution
Replace each Redis counter

Edited May 01, 2020 by Jerome Z Ng