Skip to content

Add support for multiple Redises

Problem

As part of gitlab-com/gl-infra&579 (closed) and related work, it is currently difficult to test any changes that assume data to reside in more than one Redis instance.

It could be useful to support launching more than one redis container.

The GDK supports this naturally since it is aware of per-cluster redis configs: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/be53088ad542c7a1e738c64d4e4811c23ca7ca1f/lib/gdk/command/diff_config.rb

The application will prioritize those over a generic resque.yml: https://gitlab.com/gitlab-org/gitlab/blob/1c534adfb1a60e25bc18079a7fe5eb013bf3ea37/lib/gitlab/redis/wrapper.rb#L70

The GCK always emits a single resque.yml instead.

Suggestion

I don't think we necessarily need to support N different redis instances. For testing it might suffice to just have one additional instance, similar to how we define a postgres and postgres-replica service.

That said, we could define the following compose services:

  • redis (always starts)
  • redis-alt (optionally starts)

The existing redis service would remain the default instance and be the destination for any Ruby redis wrappers for which there exists no redis.<store>.yml.

For redis-alt, we can leverage environment variables to emit config for additional kinds of stores. This env var could be defined in gck.env by the developer (e.g. GCK_REDIS_ALT_STORE_NAME=sessions). The build scripts would check for this env var and generate a matching redis.sessions.yml file that points to redis-alt:

/scripts/helpers/merge-yaml.rb /dev/stdin /home/git/gck-custom.yml:$REDIS_ALT_YML <<EOF | sponge config/$REDIS_ALT_YML
production: &production
  url: redis://redis-alt:6379
development: *production
test: *production
EOF