Skip to content

Registry: support defining a local set of sentinels

Jaime Martinez requested to merge registry-support-local-redis-sentinels into master

What does this MR do?

We recently realised that in order to expose the Reds sentinels specifically for the Registry we need to add the local settings for sentinels.

See gitlab-com/gl-infra/k8s-workloads/gitlab-com!2042 (comment 1066892795).

Testing

Assuming a cluster is ready to be used, the change can be tested with the following my-values.yml file:

USER-SUPPLIED VALUES:
global:
  redis:
    host: global.example.com
    sentinels:
      - host: global1.example.com
        port: 16379
      - host: global2.example.com
        port: 16379
    password:
      enabled: true
  registry:
    bucket: registry
registry:
  # auth: 
  database:
    enabled: true
  redis:
    cache:
      enabled: true
      host: redis.example.com
      sentinels:
        - host: sentinel1.example.com
          port: 26379
        - host: sentinel2.example.com
          port: 26379
      password:
        secret: redis-registry-cache-password-v1
        key: redis-password
      dialtimeout: 5s
      readtimeout: 5s
      writetimeout: 5s
      pool:
        size: 10
        maxlifetime: 1h
        idletimeout: 5m
  1. Upgrade the cluster with the --dry-run flag and output to a file out.out:
helm upgrade gitlab . --timeout 600s -f my-values.yaml --debug  --dry-run  > out.out
  1. Verify that the configuration that will be passed to the registry has the following content:
cat out.out | grep -B 4 -A 8 -i "mainName: redis\.example\.com"

    redis:
      cache:
        enabled: true
        addr:  "sentinel1.example.com:26379,sentinel2.example.com:26379"
        mainName: redis.example.com
        password: "REDIS_CACHE_PASSWORD"
        dialtimeout: 5s
        readtimeout: 5s
        writetimeout: 5s
        pool:
          size: 10
          maxlifetime: 1h
          idletimeout: 5m
  1. Verify that addr: matches the list of sentinel addresses and port as defined in registry.redis.cache.sentinels
  2. Verify that mainName corresponds to the host defined in registry.redis.cache.host

As extra steps to verify

  • Commenting registry.redis.cache.sentinels should render the the contents of global.redis.sentinels in the addr: field
cat out.out | grep -B 4 -A 8 -i "mainName: global\.example\.com"
    redis:
      cache:
        enabled: true
        addr:  "global1.example.com:16379,global2.example.com:16379"
        mainName: global.example.com
        password: "REDIS_CACHE_PASSWORD"
        dialtimeout: 5s
        readtimeout: 5s
        writetimeout: 5s
        pool:
          size: 10
          maxlifetime: 1h
          idletimeout: 5m

Related issues

Related to gitlab-org/container-registry#461 (closed) and gitlab-com/gl-infra/scalability#1862 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened
Edited by Jaime Martinez

Merge request reports