Automate Redis memory space analysis with cupcake-rdb
During a recent analysis we forked and extended a Redis RDB dump analysis tool and published it under cupcake-rdb.
In addition to documenting the new capabilities, we may also want to consider automating the analysis and making it easily available not just for groupscalability in GitLab.com / GitLab Infrastructure Team / Observability / Observability Issue Tracker but also stage groups.
We previously built out Redis analysis automation gitlab-com/gl-infra&212 (closed), in particular: https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/417, https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/360, https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/362. It should be possible to extend that with the additional tooling we've introduced.
Actions required
-
Automate the build of cupcake in CI so we don't have to build it on the Redis nodes. -
Update the bigkeys_extract
recipe to use cupcake. -
A wrapper around cupcake would be needed to handle targeting only Redis secondaries and locking execution among them. -
Looking into leveraging Consul for locking instead of the Redis primary.
-