Analyse Redis usage in Release related areas
Description
In our effort to improve Redis usage (gitlab-com/www-gitlab-com#4747 (closed)) as soon as possible we are also undertaking dedicated investigations per area to identify non-performant redis usage. Please dedicate 1-2 engineers from your team to go through in an investigative way through the different features of your area. The idea is to have issues at the end with specific severity and if possible some sort of effort estimation (to identify low hanging fruits)
Possible targets for investigation:
- Excessive calls
- Duplicate calls against Redis for the same value on the same request
- Slow calls
- Values that are too big stored in redis which might lockup during write
Good bundled information can be found here - gitlab-com/www-gitlab-com#4747 (comment 188806383)
Investigation
| Feature | Redis used? | Problematic? | Comments |
|---|---|---|---|
| Auto Deploy | No | No | This is a part of AutoDevOps script. |
| Building Docker Images | No | No | |
| Canary Deployments | - | - | This is a subsequent feature of Deploy Boards. |
| Deploy Boards | Yes | No | Rollout status uses Reactive Cache, however, the datasize is fairly small (Also I've noticed the other bug so filed in https://gitlab.com/gitlab-org/gitlab-ee/issues/12852) |
| Enviroments | Yes | No | 1. Terminal access to clusters are cached in Redis. This cache won't be inflated until user access the terminal feature. So no threat. 1. Etag cache on pages. Every time user access the page, the system invalidates the Etag Cache and create a new one. This stores about 3k bytes of JSON data. This cache is read everytime FE asks the data from environments.json endpoint. This cache won't be initiated unless user access environment page, which is low rate compared to the other major features such as Merge Requests/Issues. Etag cache will be expired after 20 minutes. |
| Feature Flags | No | No | |
| Incremental Rollout | - | - | This feature is just a combination of Scheduled job and AutoDevOps. |
| Pages | Yes | No | Sidekiq worker is used for deployment just like the other workers. This cannot be a threat to Redis |
| Releases | No | No | |
| Review Apps | - | - | This is a subsequent feature of Environment. |
Conclusion
As we can see above, there are no immediate threats to Redis from ~Release side.
Sidenote
Even though Redis is not harmed, there are some Postgresql/Gitaly offenders related to these features. It's summarized in https://gitlab.com/gitlab-org/gitlab-ce/issues/64255#note_189846339.