Setting up Redis MultiStore via config
Overview
This issue is spawned from #2059 (closed) as a follow up to the discussions in gitlab-org/gitlab!106123 (comment 1209290884). Sharding/migrating with Redis introduces unnecessary toil as most of the steps involved are copy-and-pasting MultiStore
code. See gitlab-org/gitlab!106123 (merged) for example on the amount of work needed.
Objective: Discuss and decide on how multistore should be defined in config
Proposed Format
Based on gitlab-org/gitlab!106123 (comment 1209379747). It introduces a new key: multi_store
.
<env>:
<store name>:
multi_store:
<primary/secondary>: "" # option 1, string pointing to another declaration
<primary/secondary>: # option 2, define store details within multi_store
url: ""
db: 1
New keys
-
multi_store
: Define primary and secondary store names or connection details.
Example usage
Continuing the example cited by @jacobvosmaer-gitlab in gitlab-org/gitlab!106123 (comment 1209379747)
Step 1. Define Gitlab::Redis::RepositoryCache
and update desired call sites with new singleton. Note that this only applies when sharding out a fraction of keys from an existing instance. For full migration, just the config change would suffice.
Step 2. Define Multistore-as-config
With the config below, all call sites using Gitlab::Redis::RepositoryCache
now uses MultiStore
as its redis
instead. With feature-flags defaulting to false, all operations route to the secondary store (cache) initially. Double-writes and cutover can be controlled via feature-flags like we do now.
`config/redis.yml`
---
production:
cache:
url: xxx
sentinels: [...]
repository_cache:
multi_store:
primary:
url: xxx
sentinels: [...]
secondary: cache // must point to the existing store
As shown above, a multi_store
key needs to be defined with a valid primary
and secondary
store. Instrumentation classes for stores can be generated on initialization (https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/instrumentation/redis.rb#L12 may need to include classes defined in config but not in code). For example, repository
store details will be used to dynamically generate the primary store without having to introduce a RepositoryCacheMigrator
singleton like in gitlab-org/gitlab!106123 (merged).
Note that config can be updated to define new MultiStore
s without feature-flag definition yamls since we only raise errors for missing definition files in test and development.