Draft: Add cross-slot pipeline functionality for cache migration

Sylvester Chinrequested to merge
sc1-cache-cluster-migration into master

What does this MR do and why?

This MR implements logic for migrating Gitlab::Redis::Cache from redis-cache to redis-cluster-cache. It does so via

  1. Gitlab::Redis::CrossSlot::Pipeline
  2. Patching Rails.cache to run cross-slot pipeline of get instead of mget
  3. Define MultiStore in Cache + define ClusterCache

This MR proposes a pipelined_xs function which performs pipelined operations in a redis cluster context. This function is supported in redis-rb v5 via redis-cluster-client. But for now since we are blocked on upgrades by Rails 7 (#367857 (comment 1065557960))

It creates a Future for each command, groups them into n pipeline requests (while maintaining ordering) and sends it out to each node. Each Gitlab::Patch::RedisPipline::Future object will map to the redis-rb's Future object.

TODO

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

See similar setup in !104335 (closed)

  1. Run pipeline command in gdk rails c
Click to show ClusterCache 
[7] pry(main)> futures = []
keys = ['a', 'b', 'c', 'd', 'e']
Gitlab::Instrumentation::RedisClusterValidator.allow_cross_slot_commands do
  Gitlab::Redis::ClusterCache.with do |redis|
    Gitlab::Redis::CrossSlot::Pipeline.new(redis).pipelined do |p|
      keys.each do |key|
        futures << p.get(key)
        futures << p.set('f', 1, ex: 500)
      end
    end
  end
=> ["1", "OK", "2", "OK", "3", "OK", "4", "OK", "5", "OK"]
[8] pry(main)> futures
=> [#<Gitlab::Redis::CrossSlot::Future:0x0000000136076e80 @redis_future=<Redis::Future [:get, "a"]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136076b60 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136076908 @redis_future=<Redis::Future [:get, "b"]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136076660 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136076408 @redis_future=<Redis::Future [:get, "c"]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136076138 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136075eb8 @redis_future=<Redis::Future [:get, "d"]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136075c60 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x0000000136075a08 @redis_future=<Redis::Future [:get, "e"]>>,
 #<Gitlab::Redis::CrossSlot::Future:0x00000001360757b0 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>]
[9] pry(main)> futures.map(&:value)
=> ["1", "OK", "2", "OK", "3", "OK", "4", "OK", "5", "OK"]
[10] pry(main)> Gitlab::Redis::ClusterCache.with {|r| r.ttl('f')}
=> 458
[11] pry(main)> Gitlab::Redis::ClusterCache.with {|r| r.get('f')}
=> "1"
Click to normal .pipelined 
futures = []
keys = ['a', 'b', 'c', 'd', 'e']
Gitlab::Instrumentation::RedisClusterValidator.allow_cross_slot_commands do
  Gitlab::Redis::ClusterCache.with do |redis|
    redis.pipelined do |p|
      keys.each do |key|
        futures << p.get(key)
        futures << p.set('f', 1, ex: 500)
      end
    end
  end
end

Redis::Cluster::CrossSlotPipeliningError: Cluster client couldn't send pipelining to single node. The commands include cross slot keys. ["a", "f", "b", "c", "d", "e"]
from /Users/sylvesterchin/.asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/redis-4.8.0/lib/redis/cluster.rb:83:in `call_pipeline'

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports