Reconcile snowflake Redis sentinel config on boxes

If you run the update config Ansible task, you will find a bunch of changes which are unique to the box and not tracked in our Gitter Ansible infra repo. We should reconcile these changes so we have all the necessary config tracked in our repo.

# Dry-run to see the diff of all the changes from our Ansible template to snowflake on the boxes
ansible-playbook -i prod playbooks/redis.yml -t redis_sentinel_update_config --diff --check

This issue was spawned from the outage https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/9416 and our desire to increase the down thresholds -> https://gitlab.com/gitlab-com/gl-infra/gitter-infrastructure/-/merge_requests/178

Current Ansible template for Redis sentinel config

https://gitlab.com/gitlab-com/gl-infra/gitter-infrastructure/-/blob/64527c2de3c4561096dbcf8b998c1ecc5492d2f4/ansible/roles/redis/sentinel/templates/sentinel.conf.j2

Current snowflake: /etc/redis/sentinel.conf from sentinel-01 

deployer@sentinel-01:~$ cat /etc/redis/sentinel.conf
protected-mode no

sentinel myid d020820076b08590d8a9a6b7edd3be0579e07ba6
sentinel monitor gitter-nopersist-prod 10.0.1.137 6381 2
sentinel down-after-milliseconds gitter-nopersist-prod 5000
sentinel failover-timeout gitter-nopersist-prod 30000
sentinel notification-script gitter-nopersist-prod /etc/redis/sentinel-notify.sh

sentinel config-epoch gitter-nopersist-prod 113
sentinel leader-epoch gitter-nopersist-prod 145
sentinel known-slave gitter-nopersist-prod 10.0.11.124 6381
sentinel known-sentinel gitter-nopersist-prod 10.0.11.224 26379 6045a11ab1f90e1dc41be87f6c0de95dbbb24854
sentinel known-sentinel gitter-nopersist-prod 10.0.21.133 26379 6b42b1c8134a41707653abb24efda700dec851a4

sentinel known-sentinel gitter-nopersist-prod 10.0.21.201 26379 2fb462a35b6a50320b9428b9f0b76466af9d9894
sentinel monitor gitter-faye-prod 10.0.11.124 6382 2
sentinel down-after-milliseconds gitter-faye-prod 5000
sentinel failover-timeout gitter-faye-prod 30000
sentinel notification-script gitter-faye-prod /etc/redis/sentinel-notify.sh

sentinel config-epoch gitter-faye-prod 142
sentinel leader-epoch gitter-faye-prod 147
sentinel known-slave gitter-faye-prod 10.0.1.137 6382
sentinel known-sentinel gitter-faye-prod 10.0.11.224 26379 6045a11ab1f90e1dc41be87f6c0de95dbbb24854
sentinel known-sentinel gitter-faye-prod 10.0.21.133 26379 6b42b1c8134a41707653abb24efda700dec851a4
# Generated by CONFIG REWRITE
port 26379
dir "/"
maxclients 4064
sentinel known-sentinel gitter-faye-prod 10.0.21.201 26379 2fb462a35b6a50320b9428b9f0b76466af9d9894
sentinel monitor gitter-caching-prod 10.0.1.137 6380 2
sentinel down-after-milliseconds gitter-caching-prod 5000
sentinel failover-timeout gitter-caching-prod 30000
sentinel notification-script gitter-caching-prod /etc/redis/sentinel-notify.sh
sentinel config-epoch gitter-caching-prod 139

sentinel leader-epoch gitter-caching-prod 144

sentinel known-slave gitter-caching-prod 10.0.11.124 6380
sentinel known-sentinel gitter-caching-prod 10.0.11.224 26379 6045a11ab1f90e1dc41be87f6c0de95dbbb24854
sentinel known-sentinel gitter-caching-prod 10.0.21.133 26379 6b42b1c8134a41707653abb24efda700dec851a4
sentinel known-sentinel gitter-caching-prod 10.0.21.201 26379 2fb462a35b6a50320b9428b9f0b76466af9d9894
sentinel monitor gitter-master 10.0.1.137 6379 2
sentinel down-after-milliseconds gitter-master 5000
sentinel failover-timeout gitter-master 30000
sentinel notification-script gitter-master /etc/redis/sentinel-notify.sh
sentinel config-epoch gitter-master 147
sentinel leader-epoch gitter-master 147
sentinel known-slave gitter-master 10.0.11.124 6379
sentinel known-sentinel gitter-master 10.0.11.224 26379 6045a11ab1f90e1dc41be87f6c0de95dbbb24854
sentinel known-sentinel gitter-master 10.0.21.133 26379 6b42b1c8134a41707653abb24efda700dec851a4
sentinel known-sentinel gitter-master 10.0.21.201 26379 2fb462a35b6a50320b9428b9f0b76466af9d9894
sentinel current-epoch 147

cc @viktomas

Edited by Eric Eastwood