Analyse Redis usage in Release related areas

Description

In our effort to improve Redis usage (gitlab-com/www-gitlab-com#4747 (closed)) as soon as possible we are also undertaking dedicated investigations per area to identify non-performant redis usage. Please dedicate 1-2 engineers from your team to go through in an investigative way through the different features of your area. The idea is to have issues at the end with specific severity and if possible some sort of effort estimation (to identify low hanging fruits)

Possible targets for investigation:

  • Excessive calls
  • Duplicate calls against Redis for the same value on the same request
  • Slow calls
  • Values that are too big stored in redis which might lockup during write

Good bundled information can be found here - gitlab-com/www-gitlab-com#4747 (comment 188806383)

Investigation

Feature Redis used? Problematic? Comments
Auto Deploy No No This is a part of AutoDevOps script.
Building Docker Images No No
Canary Deployments - - This is a subsequent feature of Deploy Boards.
Deploy Boards Yes No Rollout status uses Reactive Cache, however, the datasize is fairly small (Also I've noticed the other bug so filed in https://gitlab.com/gitlab-org/gitlab-ee/issues/12852)
Enviroments Yes No 1. Terminal access to clusters are cached in Redis. This cache won't be inflated until user access the terminal feature. So no threat.
1. Etag cache on pages. Every time user access the page, the system invalidates the Etag Cache and create a new one. This stores about 3k bytes of JSON data. This cache is read everytime FE asks the data from environments.json endpoint. This cache won't be initiated unless user access environment page, which is low rate compared to the other major features such as Merge Requests/Issues. Etag cache will be expired after 20 minutes.
Feature Flags No No
Incremental Rollout - - This feature is just a combination of Scheduled job and AutoDevOps.
Pages Yes No Sidekiq worker is used for deployment just like the other workers. This cannot be a threat to Redis
Releases No No
Review Apps - - This is a subsequent feature of Environment.

Conclusion

As we can see above, there are no immediate threats to Redis from ~Release side.

Sidenote

Even though Redis is not harmed, there are some Postgresql/Gitaly offenders related to these features. It's summarized in https://gitlab.com/gitlab-org/gitlab-ce/issues/64255#note_189846339.

Edited by Shinya Maeda