Use `Redis` to implement distributed cache of API requests
Currently, each of the processes does use API, and in-memory caching of data. We should consider adding `Redis` to Pages to: 1. have very low in-memory cache (a few seconds), like a `L1` cache 2. have a longer Redis-based cache (a few minutes), like a `L2` cache 3. possibly allow to invalidate `L2` cache by Rails on pages content change, reducing time needed to discover the config change 4. the `L1` cache could be invalidated by sending from Rails the `PubSub` notification that would be consumed by `Pages`, similarly how it is implemented on `Workhorse` This would make the: 1. much more cache-efficient, as cache would be shared between distributed nodes 2. it is cheap to cache on Redis a compressed definition of a domain, given that it is fetch of a single key 3. allow us to invalidate `L2` cache, and optionally `L1` cache 4. possibly make Redis-based cache to be even like 60 minutes, to effectively be mostly immune for intermediate disruptions to GitLab API for frequently accessed pages (within a 60 minutes window?)
issue