Use `Redis` to implement distributed cache of API requests
Currently, each of the processes does use API, and in-memory caching of data.
We should consider adding Redis
to Pages to:
- have very low in-memory cache (a few seconds), like a
L1
cache - have a longer Redis-based cache (a few minutes), like a
L2
cache - possibly allow to invalidate
L2
cache by Rails on pages content change, reducing time needed to discover the config change - the
L1
cache could be invalidated by sending from Rails thePubSub
notification that would be consumed byPages
, similarly how it is implemented onWorkhorse
This would make the:
- much more cache-efficient, as cache would be shared between distributed nodes
- it is cheap to cache on Redis a compressed definition of a domain, given that it is fetch of a single key
- allow us to invalidate
L2
cache, and optionallyL1
cache - possibly make Redis-based cache to be even like 60 minutes, to effectively be mostly immune for intermediate disruptions to GitLab API for frequently accessed pages (within a 60 minutes window?)
Edited by Kamil Trzciński