Enable GitLab to connect with Redis over TLS

What does this MR do?

Followup from gitlab!131867 (merged). MR does three things

1. Allow Redis to run only over TLS and not non-TLS port or socket. Ensure the running_version code contacts Redis over TLS.
2. Allow GitLab rails to contact such a Redis, over TLS - add settings that will be used to populate resque.yml
3. Make gitlab-ctl get-redis-master use this information to find out the master node.
4. Make gitlab-redis-cli work with TLS (only on Redis nodes)

Testing

Note: The following setup, including certificates, can be found at https://gitlab.com/balasankarc/omnibus-gitlab-docker-compose/-/tree/main/redis/single-sentinel-node/ssl

1. Setup a GitLab instance using external Redis, connected over SSL (pass the image from the latest Trigger:docker job as GITLAB_IMAGE variable)

   docker-compose.yml

   version: '3.7'
   
   x-default_settings: &default_settings
     image: "${GITLAB_IMAGE}"
     privileged: true
     shm_size: '256m'
     restart: always
     volumes:
       - "./ssl:/etc/gitlab/ssl"
       - "./trusted-certs:/etc/gitlab/trusted-certs"
   
   services:
     redis-primary-sentinel:
       <<: *default_settings
       environment:
         GITLAB_OMNIBUS_CONFIG: |
           roles ['redis_sentinel_role', 'redis_master_role']
   
           # Redis configuration
           redis['bind'] = '10.0.0.2'
           redis['tls_port'] = 6380
           redis['tls_cert_file'] = '/etc/gitlab/ssl/self_signed.crt'
           redis['tls_key_file'] = '/etc/gitlab/ssl/self_signed.key'
           redis['tls_replication'] = 'yes'
           redis['tls_auth_clients'] = 'yes'
           redis['password'] = 'toomanysecrets'
   
           # Sentinel Configuration
           sentinel['bind'] = '10.0.0.2'
           sentinel['quorum'] = 1
           sentinel['tls_port'] = 26380
           sentinel['tls_cert_file'] = '/etc/gitlab/ssl/self_signed.crt'
           sentinel['tls_key_file'] = '/etc/gitlab/ssl/self_signed.key'
           sentinel['tls_replication'] = 'yes'
           sentinel['tls_auth_clients'] = 'yes'
   
           # Information about the primary
           redis['master_ip'] = '10.0.0.2'
           redis['master_port'] = '6380'
           redis['master_name'] = 'gitlab-redis'
           redis['master_password'] = 'toomanysecrets'
   
           # Disable Redis and Sentinel non-TLS ports
           redis['port'] = 0
           sentinel['port'] = 0
   
           # This is a pure Redis node. We don't need Rails stuff.
           gitlab_rails['enable'] = false
       networks:
         default:
           ipv4_address: 10.0.0.2
   
     gitlab:
     <<: *default_settings
     environment:
       GITLAB_OMNIBUS_CONFIG: |
         external_url "http://127.0.0.1"
   
         # Redis related information
         redis['enable'] = false
         redis['master_name'] = 'gitlab-redis'
         redis['master_password'] = 'toomanysecrets'
         gitlab_rails['redis_sentinels'] = [
           { 'host' => '10.0.0.2', 'port' => 26380 },
         ]
         gitlab_rails['redis_ssl'] = true
         gitlab_rails['redis_tls_client_cert_file'] = '/etc/gitlab/ssl/self_signed.crt'
         gitlab_rails['redis_tls_client_key_file'] = '/etc/gitlab/ssl/self_signed.key'
   
         # KAS and Workhorse are known not to work with client certificates. Disable them, as we don't need them for testing.
         gitlab_workhorse['enable'] = false
         gitlab_kas['enable'] = false
     ports:
       - "80:80"
       - "443:443"
       - "22:22"
     networks:
       default:
         ipv4_address: 10.0.0.3
   
   networks:
     default:
       ipam:
         config:
           - subnet: 10.0.0.0/24

   1. We will need a self-signed certificate with 10.0.0.2 and 10.0.0.3 as SAN. Put the leaf, intermediate and root certificates to trusted-certs directory. Put the leaf certificate and key to ssl directory so they get mounted to /etc/gitlab/trusted-certs and /etc/gitlab/ssl respectively.

   2. Deploy the instance

      $ GITLAB_IMAGE="<image_url>" docker-compose up -d

2. Confirm that gitlab-ctl get-redis-master works over SSL

   $ docker-compose exec -it gitlab gitlab-ctl get-redis-master
   Redis master found at host 10.0.0.2 listening on port 6380

3. Confirm that GitLab can talk with Redis over SSL

   $ docker-compose exec -it gitlab gitlab-rails runner "redis = Redis.new(Gitlab::Redis::SharedState.params); puts redis.info['redis_version'];"
   7.0.13

4. Confirm that gitlab-redis-cli works

   $ docker-compose exec -it redis-primary-sentinel gitlab-redis-cli ping
   PONG

Related issues

Closes #8217 (closed)

Closes #6628 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• MR title and description are up to date, accurate, and descriptive.
• MR targeting the appropriate branch.
• Latest Merge Result pipeline is green.
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

• The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
• If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
• If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
• If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes.
• Documentation created/updated.
• Tests added.
• Integration tests added to GitLab QA.
• Equivalent MR/issue for the GitLab Chart opened.
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.