Skip to content

Use multistore pool for RepositoryCache.cache_store

Sylvester Chin requested to merge sc1-fix-repo-cache-multistore into master

What does this MR do and why?

This MR uses multistore_pool from the Gitlab::Redis::MultiStoreWrapper in Gitlab::Redis::RepositoryCache's cache_store. This was observed in gitlab-com/gl-infra/production#17613 (comment 1784180070) where commands from the Gitlab::RepositoryCache (wrapper over Gitlab::Redis::RepositoryCache.cache_store) was not sent to the migration target store.

We also introduce a MultiStoreConnectionPool which intercepts the checked out connection and borrows connection if it is a MultiStore before yielding.

See gitlab-com/gl-infra/scalability#2854 (closed)

Note that this does not affect SM users since it is behind a feature flag.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Set up config files as follow:

➜  gitlab git:(sc1-fix-repo-cache-multistore) cat config/redis.repository_cache.yml
---
development: unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket?db=2
test: unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket?db=12
➜  gitlab git:(sc1-fix-repo-cache-multistore) cat config/redis.yml
---
development:
  cluster_repository_cache: unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket?db=3

Ensure Feature.enable(:use_primary_and_secondary_stores_for_repository_cache) is run before proceeding.

  1. On master branch, the Gitlab::Redis::RepositoryCache.pool is used and no FF lookup nor dual-write occurs.
[1] pry(main)> Gitlab::Redis::RepositoryCache.cache_store.write('bug', 123)
=> "OK"

➜  gitlab git:(sc1-fix-repo-cache-multistore) gdk redis-cli -n 3 get cache:gitlab:bug
(nil)
➜  gitlab git:(master) gdk redis-cli -n 2 get cache:gitlab:bug
"\x04\bo: ActiveSupport::Cache::Entry\t:\x0b@valuei\x01{:\r@version0:\x10@created_atf\x060:\x10@expires_inf\x161708608454.519594"
  1. On this MR, the multistore pool is used, leading to a feature flag lookup and dual-write.
[1] pry(main)> Gitlab::Redis::RepositoryCache.cache_store.write('testingfix', 123)
  Feature::FlipperGate Pluck (1.4ms)  SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'use_primary_and_secondary_stores_for_repository_cache' /*application:console,db_config_name:main,console_hostname:SylvestersMBP2.localdomain,console_username:sylvesterchin,line:/lib/feature.rb:314:in `block in current_feature_value'*/
  Feature::FlipperGate Pluck (0.2ms)  SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'use_primary_store_as_default_for_repository_cache' /*application:console,db_config_name:main,console_hostname:SylvestersMBP2.localdomain,console_username:sylvesterchin,line:/lib/feature.rb:314:in `block in current_feature_value'*/
=> "OK"

# on terminal
➜  gitlab git:(sc1-fix-repo-cache-multistore) gdk redis-cli -n 2 get cache:gitlab:testingfix
"\x04\bo: ActiveSupport::Cache::Entry\t:\x0b@valuei\x01{:\r@version0:\x10@created_atf\x060:\x10@expires_inf\x161708608343.455152"
➜  gitlab git:(sc1-fix-repo-cache-multistore) gdk redis-cli -n 3 get cache:gitlab:testingfix
"\x04\bo: ActiveSupport::Cache::Entry\t:\x0b@valuei\x01{:\r@version0:\x10@created_atf\x060:\x10@expires_inf\x161708608343.455152"
Edited by Sylvester Chin

Merge request reports