Estimate impact of refactoring multi-key commands into multiple single-key commands
We will be moving redis-cache into a Redis Cluster setup and Rails.cache will be migrated as a result. Unfortunately, Rails.cache does not support Redis Cluster directly since multi-key operations are permitted. Multi-key operations are prone to cross-slot errors unless hash-tags are used.
Within Rails.cache, there are some components utilising multi-key operations (rg '.fetch_multi\(' and rg '.read_multi\(')
- Namespace uses hash-tags: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/namespace.rb#L720
- Count services do not use hash-tags: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/projects/batch_count_service.rb#L17
- banzai renderer does not use hash-tags: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/banzai/renderer.rb#L112
- preloader (read_multi) - https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/repository_cache/preloader.rb#L25
- cache helper - https://gitlab.com/gitlab-org/gitlab/-/blob/61628c87f299a04712c39d38c9ce0c0dfd2af3d4/lib/gitlab/cache/helpers.rb#L45-49
Since there are very few instances of multi-key operations being performed by Rails.cache, we could break up the commands into multiple single key ops. E.g. mget into several get.
The proposal is to roll out this change but control the switch using a feature-flag. This will allow us to observe the impact on apdex for redis-cache and stage groups.
See #1992 (closed)