Skip to content

feat: add support for configuring Redis Sentinels with GitLab Exporter

Stan Hu requested to merge sh-redis-sentiel-redis-exporter into master

What does this MR do?

This requires GitLab Exporter 15.0.0:

Related issues

Relates to #3813 (closed)

Sentinel passwords need to be supported in #2902 (closed) too.

Testing

I created a separate Redis and Redis Sentinel deployment:

redis-deployment.yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:latest
        ports:
        - containerPort: 6379
        env:
        - name: REDIS_PASSWORD
          value: "<password from gitlab-redis-secret secret>"
        command:
        - sh
        - -c
        - |
          echo "user default on >${REDIS_PASSWORD} ~* +@all" > /tmp/redis.conf
          redis-server /tmp/redis.conf
---
apiVersion: v1
kind: Service
metadata:
  name: redis
spec:
  ports:
  - port: 6379
    targetPort: 6379
  selector:
    app: redis

sentinel-deployment.yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-sentinel
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis-sentinel
  template:
    metadata:
      labels:
        app: redis-sentinel
    spec:
      containers:
      - name: redis-sentinel
        image: bitnami/redis-sentinel:latest
        ports:
        - containerPort: 26379
        env:
        - name: REDIS_PASSWORD
          value: "<password from gitlab-redis-secret secret>"
        command:
        - sh
        - -c
        - |
          echo "Starting Redis Sentinel..."
          cat <<EOF > /opt/bitnami/redis-sentinel/etc/sentinel.conf
          sentinel monitor mymaster redis.default.svc.cluster.local 6379 2
          sentinel down-after-milliseconds mymaster 5000
          sentinel failover-timeout mymaster 60000
          sentinel parallel-syncs mymaster 1
          sentinel auth-pass mymaster ${REDIS_PASSWORD}
          sentinel resolve-hostnames yes
          EOF
          redis-sentinel /opt/bitnami/redis-sentinel/etc/sentinel.conf
---
apiVersion: v1
kind: Service
metadata:
  name: redis-sentinel
spec:
  ports:
  - port: 26379
    targetPort: 26379
  selector:
    app: redis-sentinel

Then I ran:

kubectl apply -f redis-deployment.yaml
kubectl apply -f sentinel-deployment.yaml

In values.yaml, I updated this:

global:
  redis:
    auth:
      enabled: true
      # secret:
      # key:
    host: mymaster
    port: 6379
    # user: webservice
    sentinels:
       - host: 'redis-sentinel.default.svc.cluster.local'
         port: 26379
gitlab:
  gitlab-exporter:
    image:
      tag: 37fc057796cf29f9b34371c5562fe72ff1c02ec1

Then I logged into gitlab-exporter. I checked that /etc/gitlab-exporter/gitlab-exporter.yml had the redis_sentinels config, and made sure that the Sidekiq data was shown:

git@gitlab-gitlab-exporter-7f987f89d-52dtj:/$ curl -s http://localhost:9168/metrics | grep sidekiq
sidekiq_queue_size{name="default"} 0
sidekiq_queue_size{name="mailers"} 0
sidekiq_queue_latency_seconds{name="default"} 0
sidekiq_queue_latency_seconds{name="mailers"} 0
sidekiq_queue_paused{name="default"} 0
sidekiq_queue_paused{name="mailers"} 0
sidekiq_jobs_processed_total 355
sidekiq_jobs_failed_total 0
sidekiq_jobs_enqueued_size 0
sidekiq_jobs_scheduled_size 0
sidekiq_jobs_retry_size 0
sidekiq_jobs_dead_size 0
sidekiq_default_queue_latency_seconds 0
sidekiq_processes_size 2
sidekiq_workers_size 0

Author checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • When ready for review, follow the instructions in the "Reviewer Roulette" section of the Danger Bot MR comment, as per the Distribution experimental MR workflow

For merge requests from forks, consider the following options for Danger to work properly:

Expected (please provide an explanation if not completing)

Edited by Stan Hu

Merge request reports