Skip to content

Add ClusterRepositoryCache class for migration

Sylvester Chin requested to merge sc1-migrate-repository-cache into master

What does this MR do and why?

This MR adds ClusterRepositoryCache class for migrating repository cache workload from redis-repository-cache (a standalone Redis with sentinels) to a Redis Cluster for gitlab.com.

Gitlab::Redis::ClusterRepositoryCache will connect to a Redis Cluster for .com deployment and help migrate repository cache workload from Gitlab::Redis::RepositoryCache.

See issue: gitlab-com/gl-infra/scalability#2854 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. For SM users without an external Redis for repository cache, i.e. no config/redis.repository_cache.yml, we can verify that this change does not affect them at all by
rm config/redis.repository_cache.yml # use gdk reconfigure to restore
gdk rails c
  1. The 2 stores Cache and RepositoryCache should read/write to the same store.
[1] pry(main)> Gitlab::Redis::Cache.params
=> {:cluster=>["redis://localhost:6001"], :instrumentation_class=>Gitlab::Instrumentation::Redis::Cache, :db=>0}
[2] pry(main)> Gitlab::Redis::RepositoryCache.params
=> {:cluster=>["redis://localhost:6001"], :instrumentation_class=>Gitlab::Instrumentation::Redis::RepositoryCache, :db=>0}
[6] pry(main)> Gitlab::Redis::RepositoryCache.with {|c| c.set('testrepocache', '12345') }
=> "OK"
[7] pry(main)> Gitlab::Redis::Cache.with {|c| c.get('testrepocache') }
  1. ClusterRepositoryCache should have the same params and read the same value, i.e., if only config/redis.cache.yml is defined, all 3 stores are effectively identical.
[8] pry(main)> Gitlab::Redis::ClusterRepositoryCache.with {|c| c.get('testrepocache') }
=> "12345"
[9] pry(main)> Gitlab::Redis::ClusterRepositoryCache.params
=> {:cluster=>["redis://localhost:6001"], :instrumentation_class=>Gitlab::Instrumentation::Redis::ClusterRepositoryCache, :db=>0}
  1. The MultiStore does not affect SM users too since it defaults to the secondary store if same_redis_store? is true.
[11] pry(main)> Gitlab::Redis::RepositoryCache.multistore.send(:same_redis_store?)
=> true

For gitlab.com:

We have config/redis.cache.yml and config/redis.repository_cache.yml configured. Since we have not provisioned a Cluster for ClusterRepositoryCache yet, the missing configuration will mean it uses Cache's details. However, as long as the feature-flag is disabled, it will not be used.

  1. Run gdk reconfigure and update config/redis.repository_cache.yml to use a different db from cache
  gitlab git:(sc1-migrate-repository-cache) cat config/redis.repository_cache.yml
---
development: unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket?db=3
test: unix:/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket?db=12
  1. Verify using gdk rails c that RepositoryCache has a different configuration:
[1] pry(main)> Gitlab::Redis::RepositoryCache.params
=> {:instrumentation_class=>Gitlab::Instrumentation::Redis::RepositoryCache, :path=>"/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket", :db=>3}
[2] pry(main)> Gitlab::Redis::ClusterRepositoryCache.params
=> {:instrumentation_class=>Gitlab::Instrumentation::Redis::ClusterRepositoryCache, :path=>"/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket", :db=>2}
[3] pry(main)> Gitlab::Redis::Cache.params
=> {:instrumentation_class=>Gitlab::Instrumentation::Redis::Cache, :path=>"/Users/sylvesterchin/work/gitlab-development-kit/redis/redis.socket", :db=>2}
  1. Verify that RepositoryCache writes to a different Redis db (with both feature-flags disabled), i.e., config fallback behaviour is working fine.
[4] pry(main)> Gitlab::Redis::RepositoryCache.with {|c| c.set('testrepocache', 'abcdef') }
  Feature::FlipperGate Pluck (3.1ms)  SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'use_primary_and_secondary_stores_for_repository_cache' /*application:console,db_config_name:main,console_hostname:SylvestersMBP2.localdomain,console_username:sylvesterchin,line:/lib/feature.rb:314:in `block in current_feature_value'*/
  Feature::FlipperGate Pluck (0.3ms)  SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'use_primary_store_as_default_for_repository_cache' /*application:console,db_config_name:main,console_hostname:SylvestersMBP2.localdomain,console_username:sylvesterchin,line:/lib/feature.rb:314:in `block in current_feature_value'*/
=> "OK"
[5] pry(main)> Gitlab::Redis::ClusterRepositoryCache.with {|c| c.get('testrepocache') }
=> nil
[6] pry(main)> Gitlab::Redis::Cache.with {|c| c.get('testrepocache') }
=> nil
Edited by Sylvester Chin

Merge request reports