Skip to content

Docs - Multiple Redis instances

Ben Prescott_ requested to merge docs/bprescott/20230214-redis into master

What does this MR do?

Improvement to persistent classes table

  • Add repository_cache
  • Specify the releases which support each persistence class. This is mainly needed for repository_cache since at the time of writing, all the other classes have been supported since %14.5, but for completeness I've added versions for the previous three as well.

Running multiple Redis nodes when not using Sentinel

Cache vs. Persistent

Clarification around scaling Redis

A customer enquired about scaling Redis, and when I looked at the reference architecture docs and Redis docs it was unclear to me which instance is used depending on the supplied configuration.

In the product docs, the guidance is as follows ..

For each persistence class, GitLab defaults to using the configuration specified in gitlab_rails['redis_sentinels'] unless overridden by the previously described settings.

At the time, gitlab_rails['redis_sentinels'] was not specified in the reference architecture. It used to configure persistent and cache by specifying all four data types that fall back to default. This was fine for 'clean sheet' or 'green field' deployment, but customers with an existing Redis will want to just add the cache definition, not reconfigure how to access the existing Redis.

Since the reference architecture documentation changed since I started work on this I've now documented the previous approach for the persistent instance, since if we encounter this in customer configuration it won't be clear what it does.

The configuration docs for Rails itself added some enlightenment:

The Redis instances that can be configured are described in the table below. The order of precedence for configuration is described below, where $NAME and $FALLBACK_NAME are the upper-cased instance names from the table, and $name and $fallback_name are the lower-cased versions:

  1. The configuration file pointed to by the GITLAB_REDIS_$NAME_CONFIG_FILE environment variable.
  2. The configuration file redis.$name.yml.
  3. If a fallback instance is available, the configuration file redis.$fallback_name.yml.
  4. The configuration file pointed to by the GITLAB_REDIS_CONFIG_FILE environment variable.
  5. The configuration file resque.yml.

An example configuration file for Redis is in this directory under the name resque.yml.example.

Name Fallback instance Purpose >
cache Volatile non-persistent data >
queues Background job processing queues >
shared_state Persistent application state >
trace_chunks shared_state CI trace chunks
rate_limiting cache Rate limiting state
sessions shared_state Sessions
repository_cache cache Repository related information
db_load_balancing shared_state Database Load Balancing

Related issues

Author's checklist

If you are a GitLab team member and only adding documentation, do not add any of the following labels:

  • ~"frontend"
  • ~"backend"
  • ~"type::bug"
  • ~"database"

These labels cause the MR to be added to code verification QA issues.

Reviewer's checklist

Documentation-related MRs should be reviewed by a Technical Writer for a non-blocking review, based on Documentation Guidelines and the Style Guide.

If you aren't sure which tech writer to ask, use roulette or ask in the #docs Slack channel.

  • If the content requires it, ensure the information is reviewed by a subject matter expert.
  • Technical writer review items:
    • Ensure docs metadata is present and up-to-date.
    • Ensure the appropriate labels are added to this MR.
    • Ensure a release milestone is set.
    • If relevant to this MR, ensure content topic type principles are in use, including:
      • The headings should be something you'd do a Google search for. Instead of Default behavior, say something like Default behavior when you close an issue.
      • The headings (other than the page title) should be active. Instead of Configuring GDK, say something like Configure GDK.
      • Any task steps should be written as a numbered list.
      • If the content still needs to be edited for topic types, you can create a follow-up issue with the docs-technical-debt label.
  • Review by assigned maintainer, who can always request/require the reviews above. Maintainer's review can occur before or after a technical writer review.
Edited by Brie Carranza

Merge request reports