Skip to content

Draft: Add cross-slot pipelined API for cluster-safe pipelining

Sylvester Chin requested to merge sc1-xs-pipeline into master

What does this MR do and why?

This MR proposes a pipelined_xs function which performs pipelined operations in a redis cluster context. This function is supported in redis-rb v5 via redis-cluster-client. But for now since we are blocked on upgrades by Rails 7 (#367857 (comment 1065557960))

It creates a Future for each command, groups them into n pipeline requests (while maintaining ordering) and sends it out to each node. Each Gitlab::Patch::RedisPipline::Future object will map to the redis-rb's Future object.

irb(main):004:0> c = Redis.new(url: 'redis://127.0.0.1:6379')
=> #<Redis client v5.0.5 for redis://127.0.0.1:6379/0>
irb(main):005:0> c.set('a', 12)
=> "OK"
irb(main):006:0> f = nil
=> nil
irb(main):007:1* c.pipelined do |p|
irb(main):008:1*   f = p.get('a')
irb(main):009:0> end
=> ["12"]
irb(main):010:0> f
=> <Redis::Future [:get, "a"]>
irb(main):011:0> f.value <-- still supported in redis-rb v5
=> "12"

Follow up of !104335 (comment 1181212601)

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

See similar setup in !104335 (closed)

  1. Run pipeline command in gdk rails c 
[6] pry(main)> 
futures = []
keys = ['a', 'b', 'c', 'd', 'e']
Gitlab::Redis::CacheCluster.with do |redis|
  Gitlab::Redis::CrossSlot::Pipeline.new(redis).pipelined do |p|
    keys.each do |key|
      futures << p.get(key)
      futures << p.set('f', 1, ex: 500)
    end
  end
end

=> {"127.0.0.1:7001"=>[[:get, ["a"]], [:get, ["d"]], [:get, ["e"]]],
 "127.0.0.1:7101"=>[[:set, ["f", 1, {:ex=>500}]], [:get, ["b"]], [:set, ["f", 1, {:ex=>500}]], [:set, ["f", 1, {:ex=>500}]], [:set, ["f", 1, {:ex=>500}]], [:set, ["f", 1, {:ex=>500}]]],
 "127.0.0.1:7201"=>[[:get, ["c"]]]}
[7] pry(main)> futures
=> [#<Gitlab::Patch::RedisPipeline::Future:0x000000011eebba08 @redis_future=<Redis::Future [:get, "a"]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebb788 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebb648 @redis_future=<Redis::Future [:get, "b"]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebb418 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebb260 @redis_future=<Redis::Future [:get, "c"]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebafe0 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebaf40 @redis_future=<Redis::Future [:get, "d"]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebad38 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebac48 @redis_future=<Redis::Future [:get, "e"]>>,
 #<Gitlab::Patch::RedisPipeline::Future:0x000000011eebaa18 @redis_future=<Redis::Future [:set, "f", "1", "EX", 500]>>]
[8] pry(main)> futures.map(&:value)
=> ["a", "OK", "b", "OK", "c", "OK", "d", "OK", "e", "OK"]
[9] pry(main)> Gitlab::Redis::CacheCluster.with {|r| r.ttl('f')}
=> 429
[10] pry(main)> Gitlab::Redis::CacheCluster.with {|r| r.get('f')}
=> "1"

Compared to pipelined

[13] pry(main)> Gitlab::Redis::CacheCluster.with do |redis|
[13] pry(main)*   redis.pipelined do |p|
[13] pry(main)*     keys.each do |key|
      futures << p.get(key)
      futures << p.set('f', 1, ex: 500)
    end
  end
[13] pry(main)*     keys.each do |key|
      futures << p.get(key)
      futures << p.set('f', 1, ex: 500)
    end
  end
end
Redis::Cluster::CrossSlotPipeliningError: Cluster client couldn't send pipelining to single node. The commands include cross slot keys. ["a", "f", "b", "c", "d", "e"]
from /Users/sylvesterchin/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/redis-4.8.0/lib/redis/cluster.rb:83:in `call_pipeline'

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sylvester Chin

Merge request reports