[pre] Move repository workload from redis-cache to redis-repository-cache
Redis is up and running a functional quorum in pre:
PRODUCTION PRIMARY-REDIS stejacks@redis-repository-cache-01-db-pre.c.gitlab-pre.internal:~$ sudo gitlab-redis-cli
0.0.0.0:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=redis-repository-cache-02-db-pre.c.gitlab-pre.internal,port=6379,state=online,offset=245397,lag=0
slave1:ip=redis-repository-cache-03-db-pre.c.gitlab-pre.internal,port=6379,state=online,offset=245397,lag=0
master_failover_state:no-failover
master_replid:e5fa6ed7e75231fd1970dbadf18e0ea462057349
master_replid2:39f721e3bb0143ba968e8216934406ed012f65c4
master_repl_offset:245397
second_repl_offset:110925
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:94148
repl_backlog_histlen:151250
To move over repository traffic:
- Merge gitlab-com/gl-infra/k8s-workloads/gitlab-com!2500 (merged) to provide connection information for redis-repository-cache in helm charts.
- Start double writes:
# start double write
Feature.enable(:use_primary_and_secondary_stores_for_repository_cache)
# to check
Feature.enabled?(:use_primary_and_secondary_stores_for_repository_cache)
# rollback
Feature.disable(:use_primary_and_secondary_stores_for_repository_cache)
- Confirm keys are going to both places.
-
Sit around for a certain amount of time (not very long in pre, I suspect) and then...
-
Switch writes 100% to the new location.
# use repository cache as primary store
Feature.enable(:use_primary_store_as_default_for_repository_cache)
# to check
Feature.enabled?(:use_primary_store_as_default_for_repository_cache)
# rollback to use Cache as primary store
Feature.disable(:use_primary_store_as_default_for_repository_cache)
- Check traffic drop for redis-cache out of curiosity.
Edited by Stephanie Jackson