Skip to content

Use `Redis` to implement distributed cache of API requests

Currently, each of the processes does use API, and in-memory caching of data.

We should consider adding Redis to Pages to:

  1. have very low in-memory cache (a few seconds), like a L1 cache
  2. have a longer Redis-based cache (a few minutes), like a L2 cache
  3. possibly allow to invalidate L2 cache by Rails on pages content change, reducing time needed to discover the config change
  4. the L1 cache could be invalidated by sending from Rails the PubSub notification that would be consumed by Pages, similarly how it is implemented on Workhorse

This would make the:

  1. much more cache-efficient, as cache would be shared between distributed nodes
  2. it is cheap to cache on Redis a compressed definition of a domain, given that it is fetch of a single key
  3. allow us to invalidate L2 cache, and optionally L1 cache
  4. possibly make Redis-based cache to be even like 60 minutes, to effectively be mostly immune for intermediate disruptions to GitLab API for frequently accessed pages (within a 60 minutes window?)
Edited by Kamil Trzciński