Skip to content

Automate Redis memory space analysis with cupcake-rdb

During a recent analysis we forked and extended a Redis RDB dump analysis tool and published it under cupcake-rdb.

In addition to documenting the new capabilities, we may also want to consider automating the analysis and making it easily available not just for groupscalability in GitLab.com / GitLab Infrastructure Team / Observability / Observability Issue Tracker but also stage groups.

We previously built out Redis analysis automation gitlab-com/gl-infra&212 (closed), in particular: https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/417, https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/360, https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/362. It should be possible to extend that with the additional tooling we've introduced.

Actions required

  • Automate the build of cupcake in CI so we don't have to build it on the Redis nodes.
  • Update the bigkeys_extract recipe to use cupcake.
  • A wrapper around cupcake would be needed to handle targeting only Redis secondaries and locking execution among them.
    • Looking into leveraging Consul for locking instead of the Redis primary.
Edited by Ahmad Sherif
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information