redis: Introduce support for Redis Cluster and specifying acl user
What does this MR do?
This introduces Redis Cluster support. We are deploying Redis Cluster for GitLab.com gitlab-com/gl-infra&823 (closed), and require client configuration.
The intention is for this usage to be internal only. This is why I skipped documenting these new settings for the time being. It is also unlikely that self-managed users will need it. And we do not support it, at least not yet.
Additionally, this new code path intentionally bypasses the merging logic that we have in place for other Redises. The reason for this is two-fold:
- Redis Cluster does not use the
url
parameter, which we rely on everywhere else. It does not use ahost
parameter either. Instead, it uses separateuser
andpassword
fields. We don't want to inherit those from the top level. - Reduce surface area of the patch, effectively introducing a new case that is unlikely to break existing users.
Happy to discuss this aspect though, since Distribution may have existing policies.
Sample values.yaml
:
global:
redis:
host: resque.redis
port: 6379
clusterCache:
user: foo
password:
enabled: true
cluster:
- host: s1.cache.redis
- host: s2.cache.redis
redis:
install: false
Sample output:
redis.cluster_cache.yml.erb: |
production:
username: foo
password: <%= File.read("/etc/gitlab/redis/clusterCache-password").strip.to_json %>
cluster:
- host: s1.cache.redis
port: 6379
- host: s2.cache.redis
port: 6379
id:
We add clusterCache
and clusterRateLimiting
as new top-level keys. The use of cluster is not enforced for these keys, any redis config can be set to use cluster. The reason for these two additional targets is to allow for an online transition on GitLab.com. We may choose to consolidate these after the migration is complete.
Related issues
- gitlab-com/gl-infra/scalability#2041 (closed)
- gitlab-org/omnibus-gitlab!6548 (merged) - Equivalent omnibus change
Checklist
See Definition of done.
For anything in this list which will not be completed, please provide a reason in the MR discussion.
Required
-
Merge Request Title and Description are up to date, accurate, and descriptive -
MR targeting the appropriate branch -
MR has a green pipeline on GitLab.com -
When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow
Expected (please provide an explanation if not completing)
-
Test plan indicating conditions for success has been posted and passes -
Documentation created/updated -
Tests added -
Integration tests added to GitLab QA -
Equivalent MR/issue for omnibus-gitlab opened -
Validate potential values for new configuration settings. Formats such as integer 10
, duration10s
, URIscheme://user:passwd@host:port
may require quotation or other special handling when rendered in a template and written to a configuration file.