Introduce support for Redis Cluster and specifying acl user

What does this MR do?

This introduces Redis Cluster support. We are deploying Redis Cluster for GitLab.com gitlab-com/gl-infra&823 (closed), and require client configuration. The configuration file for each instance will either be for connecting to a (1) single Redis server via url, (2) sentinels via url + sentinels, or (3) Redis cluster via cluster.

For now, cluster support is permitted for rate_limiting and cluster_rate_limiting instances as the GitLab Rails application supports it. The intention is for this usage to be internal only. It is also unlikely that self-managed users will need it. And we do not support it, at least not yet.

This MR was written while referencing https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/doc/development/add-remove-configuration-options.md

The equivalent MR for charts is gitlab-org/charts/gitlab!2873 (merged)

Click to show sentinel config

default['gitlab']['gitlab-rails']['redis_rate_limiting_instance'] = 'redis://:REDACTED@ratelimiting-primary'
default['gitlab']['gitlab-rails']['redis_rate_limiting_sentinels'] = [{host: '127.0.0.1', port: 7001}, {host: '127.0.0.1', port: 7002}]
default['gitlab']['gitlab-rails']['redis_rate_limiting_username'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_password'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_cluster_nodes'] = []

sudo gitlab-ctl  reconfigure ...

ubuntu@rich-armadillo:~$ sudo cat  /opt/gitlab/embedded/service/gitlab-rails/config/redis.rate_limiting.yml
production:
  url: redis://:REDACTED@ratelimiting-primary
  sentinels:
    -
      host: 127.0.0.1
      port: 7001
    -
      host: 127.0.0.1
      port: 7002

Click to show invalid config

default['gitlab']['gitlab-rails']['redis_rate_limiting_instance'] = 'redis://:REDACTED@ratelimiting-primary'
default['gitlab']['gitlab-rails']['redis_rate_limiting_sentinels'] = [{host: '127.0.0.1', port: 7001}, {host: '127.0.0.1', port: 7002}]
default['gitlab']['gitlab-rails']['redis_rate_limiting_username'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_password'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_cluster_nodes'] = [{host: '127.0.0.1', port: 7001}, {host: '127.0.0.1', port: 7002}]


sudo gitlab-ctl  reconfigure ...

[2022-12-02T14:25:02+08:00] ERROR: Running exception handlers
There was an error running gitlab-ctl reconfigure:

Both sentinel and cluster configurations are defined for rate_limiting

Click to show cluster config

default['gitlab']['gitlab-rails']['redis_rate_limiting_instance'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_sentinels'] = []
default['gitlab']['gitlab-rails']['redis_rate_limiting_username'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_password'] = nil
default['gitlab']['gitlab-rails']['redis_rate_limiting_cluster_nodes'] = [{host: '127.0.0.1', port: 7001}, {host: '127.0.0.1', port: 7002}]

sudo gitlab-ctl  reconfigure ...

buntu@rich-armadillo:~$ sudo cat  /opt/gitlab/embedded/service/gitlab-rails/config/redis.rate_limiting.yml
production:
  username: rluser
  password: rlpassword
  cluster:
    -
      host: 127.0.0.1
      port: 7001
    -
      host: 127.0.0.1
      port: 7002

TODO

• documentation
• specs once desired behaviour is decided

Related issues

gitlab-com/gl-infra/scalability#2017 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

• Merge Request Title, and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• Pipeline is green on dev.gitlab.org if the change is touching anything besides documentation or internal cookbooks
• trigger-package has a green pipeline running against latest commit

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for the GitLab Chart opened