Skip to content

Add Redis settings specific to Workhorse

Balasankar 'Balu' C requested to merge refactor-redis-helper into master

What does this MR do?

Context

The relationship between Redis, GitLab Rails and GitLab Workhorse is a bit interesting. Rails and Workhorse has a line of communication with each other through Redis. Rails will write messages to specific channels, and Workhorse will subscribe to those channels. This means, both Rails and Workhorse should be pointing to the same Redis instance in their configuration files. In the past, this was controlled by two settings - gitlab_rails['redis_*'] settings, which applied to all the Redis configuration used by Rails (including individual instance configuration like Cache, SharedState, Workhorse, etc.) and to Workhorse. In addition to it, there was also gitlab_rails['redis_workhorse_*'] settings, which was used to configure just the Redis instance that Rails should use to publish Workhorse related information.

However, there was no way to specifically configure Workhorse's Redis settings without touching gitlab_rails[*] settings. This meant, !7233 (merged) was useless.

Changes

This MR add Redis settings specific to Workhorse. The precedence then becomes as follows

  1. If gitlab_workhorse['redis_*'] settings are set, they win over everything else.
    1. gitlab_rails['redis_workhorse_*'] settings are modified based on these values so that Rails and Workhorse talks to the same Redis
  2. If gitlab_rails['redis_workhorse_*'] settings are set, gitlab_workhorse['redis_*'] settings are computed from it. This retains backward compatibility.
  3. If gitlab_rails['redis_*'] settings are set, gitlab_workhorse['redis_*'] settings are computed from it. This retains backward compatibility.

While doing so, this MR also introduce a new architecture for handling Redis information of various components. However, because modifying all components to use this architecture is out of scope for this MR, we use a temporary name and location for this architecture. It will go away in future MRs, once everything is ported to this new architecture.

Testing

  • Default - no separate Redis for Workhorse

    docker-compose.yml 
    version: '3.8'
name: 'gitlab-test'

x-default_settings: &default_settings
  image: "${GITLAB_IMAGE:-registry.gitlab.com/gitlab-org/omnibus-gitlab/gitlab-ee:refactor-redis-helper}"
  pull_policy: always
  privileged: true
  shm_size: '256m'
  restart: always

services:
  gitlab:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url "http://127.0.0.1"
    ports:
      - "80:80"
      - "443:443"
      - "22:22"
    networks:
      default:
        ipv4_address: 10.0.0.2

networks:
  default:
    ipam:
      config:
        - subnet: 10.0.0.0/24
    • No /var/opt/gitlab/gitlab-rails/etc/redis.workhorse.yml file present
    • /var/opt/gitlab/gitlab-rails/etc/resque.yml and /var/opt/gitlab/gitlab-workhorse/config.toml has same information regarding Redis, pointing to the global Redis (over socket)

  • External global redis specified via gitlab_rails['redis_host']

    docker-compose.yml 
    version: '3.8'
name: 'gitlab-test'

x-default_settings: &default_settings
  image: "${GITLAB_IMAGE:-registry.gitlab.com/gitlab-org/omnibus-gitlab/gitlab-ee:refactor-redis-helper}"
  pull_policy: always
  privileged: true
  shm_size: '256m'
  restart: always

services:
  redis:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        roles ['redis_master_role']

        # Redis configuration
        redis['bind'] = '10.0.0.2'
        redis['port'] = 6379
        redis['password'] = 'toomanysecrets'

        # This is a pure Redis node. We don't need Rails stuff.
        gitlab_rails['enable'] = false
    networks:
      default:
        ipv4_address: 10.0.0.2
    healthcheck:
      test: ["CMD", "gitlab-redis-cli", "ping"]
      start_period: 60s
  gitlab:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url "http://127.0.0.1"

        redis['enable'] = false

        # Redis related information for Rails
        gitlab_rails['redis_host'] = '10.0.0.2'
        gitlab_rails['redis_port'] = '6379'
        gitlab_rails['redis_password'] = 'toomanysecrets'
    ports:
      - "80:80"
      - "443:443"
      - "22:22"
    networks:
      default:
        ipv4_address: 10.0.0.3
    healthcheck:
      disable: true
    depends_on:
      redis:
        condition: service_healthy

networks:
  default:
    ipam:
      config:
        - subnet: 10.0.0.0/24
    • No /var/opt/gitlab/gitlab-rails/etc/redis.workhorse.yml file present
    • /var/opt/gitlab/gitlab-rails/etc/resque.yml and /var/opt/gitlab/gitlab-workhorse/config.toml has same information regarding Redis, pointing to the global external Redis

  • Separate Redis for workhorse specified via gitlab_rails['redis_workhorse_*'] settings

    docker-compose.yml 
    version: '3.8'
name: 'gitlab-test'

x-default_settings: &default_settings
  image: "${GITLAB_IMAGE:-registry.gitlab.com/gitlab-org/omnibus-gitlab/gitlab-ee:refactor-redis-helper}"
  pull_policy: always
  privileged: true
  shm_size: '256m'
  restart: always

services:
  redis-for-rails:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        roles ['redis_master_role']

        # Redis configuration
        redis['bind'] = '10.0.0.2'
        redis['port'] = 6379
        redis['password'] = 'rails-redis-password'

        # This is a pure Redis node. We don't need Rails stuff.
        gitlab_rails['enable'] = false
    networks:
      default:
        ipv4_address: 10.0.0.2
    healthcheck:
      test: ["CMD", "gitlab-redis-cli", "ping"]
      start_period: 60s

  redis-for-workhorse:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        roles ['redis_master_role']

        # Redis configuration
        redis['bind'] = '10.0.0.3'
        redis['port'] = 6379
        redis['password'] = 'workhorse-redis-password'

        # This is a pure Redis node. We don't need Rails stuff.
        gitlab_rails['enable'] = false
    networks:
      default:
        ipv4_address: 10.0.0.3
    healthcheck:
      test: ["CMD", "gitlab-redis-cli", "ping"]
      start_period: 60s

  gitlab:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url "http://127.0.0.1"

        redis['enable'] = false

        # Redis related information for Rails
        gitlab_rails['redis_host'] = '10.0.0.2'
        gitlab_rails['redis_port'] = '6379'
        gitlab_rails['redis_password'] = 'rails-redis-password'

        # Redis related information for Workhorse
        gitlab_rails['redis_workhorse_instance'] = 'redis://:workhorse-redis-password@10.0.0.3/'
    ports:
      - "80:80"
      - "443:443"
      - "22:22"
    networks:
      default:
        ipv4_address: 10.0.0.4
    healthcheck:
      disable: true
    depends_on:
      redis-for-rails:
        condition: service_healthy
      redis-for-workhorse:
        condition: service_healthy

networks:
  default:
    ipam:
      config:
        - subnet: 10.0.0.0/24
    • /var/opt/gitlab/gitlab-rails/etc/redis.workhorse.yml file present
    • /var/opt/gitlab/gitlab-rails/etc/redis.workhorse.yml and /var/opt/gitlab/gitlab-workhorse/config.toml has same information regarding Redis, pointing to the Redis specific for Workhorse.
    • /var/opt/gitlab/gitlab-rails/etc/resque.yml has different information, pointing to the global Redis

  • Separate Redis for workhorse specified via gitlab_workhorse['redis_*'] settings

    docker-compose.yml 
    version: '3.8'
name: 'gitlab-test'

x-default_settings: &default_settings
  image: "${GITLAB_IMAGE:-registry.gitlab.com/gitlab-org/omnibus-gitlab/gitlab-ee:refactor-redis-helper}"
  pull_policy: always
  privileged: true
  shm_size: '256m'
  restart: always

services:
  redis-for-rails:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        roles ['redis_master_role']

        # Redis configuration
        redis['bind'] = '10.0.0.2'
        redis['port'] = 6379
        redis['password'] = 'rails-redis-password'

        # This is a pure Redis node. We don't need Rails stuff.
        gitlab_rails['enable'] = false
    networks:
      default:
        ipv4_address: 10.0.0.2
    healthcheck:
      test: ["CMD", "gitlab-redis-cli", "ping"]
      start_period: 60s

  redis-for-workhorse:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        roles ['redis_master_role']

        # Redis configuration
        redis['bind'] = '10.0.0.3'
        redis['port'] = 6379
        redis['password'] = 'workhorse-redis-password'

        # This is a pure Redis node. We don't need Rails stuff.
        gitlab_rails['enable'] = false
    networks:
      default:
        ipv4_address: 10.0.0.3
    healthcheck:
      test: ["CMD", "gitlab-redis-cli", "ping"]
      start_period: 60s

  gitlab:
    <<: *default_settings
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url "http://127.0.0.1"

        redis['enable'] = false

        # Redis related information for Rails
        gitlab_rails['redis_host'] = '10.0.0.2'
        gitlab_rails['redis_port'] = '6379'
        gitlab_rails['redis_password'] = 'rails-redis-password'

        # Redis related information for Workhorse
        gitlab_workhorse['redis_host'] = '10.0.0.3'
        gitlab_workhorse['redis_port'] = '6379'
        gitlab_workhorse['redis_password'] = 'workhorse-redis-password'
    ports:
      - "80:80"
      - "443:443"
      - "22:22"
    networks:
      default:
        ipv4_address: 10.0.0.4
    healthcheck:
      disable: true
    depends_on:
      redis-for-rails:
        condition: service_healthy
      redis-for-workhorse:
        condition: service_healthy

networks:
  default:
    ipam:
      config:
        - subnet: 10.0.0.0/24
    • /var/opt/gitlab/gitlab-rails/etc/redis.workhorse.yml file present
    • /var/opt/gitlab/gitlab-rails/etc/redis.workhorse.yml and /var/opt/gitlab/gitlab-workhorse/config.toml has same information regarding Redis, pointing to the Redis specific for Workhorse.
    • /var/opt/gitlab/gitlab-rails/etc/resque.yml has different information, pointing to the global Redis

Related issues

Closes: #8300 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes.
  • Documentation created/updated.
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened.
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Robert Marshall

Merge request reports