Integrate Redis Cluster into Gitlab's CI pipeline
GitLab application runs against a single Redis instance (no sentinel) in the CI pipelines.
- set-up script prepares a single config file to be used by all
Gitlab::Redis::xxx
objects(https://gitlab.com/gitlab-org/gitlab/-/blob/2fcdd3c600136d8a9e28e8453b8869c96fb8b6d2/scripts/prepare_build.sh#L54) - https://gitlab.com/gitlab-org/gitlab/-/blob/2fcdd3c600136d8a9e28e8453b8869c96fb8b6d2/.gitlab/ci/global.gitlab-ci.yml#L245
The main purpose of this is to ensure future use cases for Gitlab::Redis::Cache
remain compatible with Redis Cluster. i.e. if a dev adds mget
without using hash-tags after &878 (closed) is wrapped up, the CI pipeline will catch it and not let it be deployed where mget
will fail, necessitating a rollback.
Proposal
Phase 1
We should start running Gitlab::Redis::Cache
against a Redis Cluster container after #1992 (closed) is added to the Gitlab Rails codebase.
Add a .use-redis-cluster
deployment to CI via https://hub.docker.com/r/grokzen/redis-cluster.
As for coverage, the plan is to use both Redis and Redis Cluster (where appropriate) for specs by default. Redis-only jobs should exist but it does not need to cover all variations (pg12, pg13 and pg14).
Phase 2
Building our own container on gitlab-build-images (e.g. redis-cluster-7.0)