Rails now fails to configure with being configured to point to multiple HA Redis clusters
Summary
One of our test environments failed today with the following error on Rails:
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" ----
STDOUT: rake aborted!
ArgumentError: RedisClient::SentinelConfig requires either a name or an url with a host
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:102:in `new'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:102:in `init_redis'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:93:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:38:in `block in pool'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:26:in `with'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/7_redis.rb:38:in `block in <top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/7_redis.rb:37:in `each'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/7_redis.rb:37:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:7:in `<top (required)>'
<internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" ----
Ran "bash" returned 1
This environment built fine yesterday and no config was changed but it is deploying the latest Omnibus nightly.
A MR was recently merged that refactored Redis config in Omnibus that could be a factor here.
For reference the redacted config we use in the environment for Rails to connect to Redis is as follows:
# Components Config
# Redis
### Multi Node - Separate Queues
#### Default
redis['master_name'] = 'gitlab-redis-persistent'
redis['master_password'] = '<redacted>'
gitlab_rails['redis_sentinels'] = [
{host: '<redacted>', port: 26379},
{host: '<redacted>', port: 26379},
{host: '<redacted>', port: 26379},
]
#### Cache
gitlab_rails['redis_cache_instance'] = 'redis://:<redacted>@gitlab-redis-cache'
gitlab_rails['redis_cache_sentinels'] = [
{host: '<redacted>', port: 26379},
{host: '<redacted>', port: 26379},
{host: '<redacted>', port: 26379},
]
/label ~"type::bug"