Add support for setting TLS parameters

Stan Hurequested to merge
sh-add-mutual-tls into master

What does this MR do?

This adds support for setting TLS parameters in the Rails config. This makes it possible to use Redis Sentinel with TLS.

Related issues

Relates to #6422

How to test locally

  1. Start up a Sentinel cluster and Redis master:
git clone https://gitlab.com/stanhu/redis-sentinel-setup.git
cd redis-sentinel-setup
./generate-certs.sh
chmod a+r tls/*
docker-compose up

This starts up a container named redis-master with a Redis master name of mymaster.

  1. Add a temporary entry to /etc/hosts:
127.0.0.1   redis-master
  1. Copy over the certs into /etc/gitlab/trusted-certs:
cp tls/*.crt /etc/gitlab/trusted-certs
  1. Copy the keys into /etc/gitlab/ssl:
cp tls/*.key /etc/gitlab/ssl
  1. In `/etc/gitlab/gitlab.rb, add:
redis['master_name'] = 'mymaster'
gitlab_rails['redis_sentinels'] = [
  { host: "localhost", port: "26379" },
  { host: "localhost", port: "26380" }
]
gitlab_rails['redis_sentinels_ssl'] = true
gitlab_rails['redis_ssl'] = true
gitlab_exporter['probe_sidekiq'] = true

gitlab_rails['redis_tls_ca_cert_file'] = '/etc/gitlab/ssl/redis-bundle.crt'
gitlab_rails['redis_tls_client_cert_file'] = '/etc/gitlab/ssl/redis-client.crt'
gitlab_rails['redis_tls_client_key_file'] = '/etc/gitlab/ssl/redis-client.key'

gitlab_rails['redis_sentinels_tls_ca_cert_file'] = '/etc/gitlab/ssl/sentinel-bundle.crt'
gitlab_rails['redis_sentinels_tls_client_cert_file'] = '/etc/gitlab/ssl/redis-client.crt'
gitlab_rails['redis_sentinels_tls_client_key_file'] = '/etc/gitlab/ssl/redis-client.key'
  1. Run gitlab-ctl reconfigure and ensure Rails is up, and the config files in /var/opt/gitlab/gitlab-rails/etc have Sentinel configs and ssl: true:
$ cat /var/opt/gitlab/gitlab-rails/etc/resque.yml
production:
  url: rediss://mymaster/
  secret_file: /var/opt/gitlab/gitlab-rails/shared/encrypted_settings/redis.yml.enc
  ssl: true
  ssl_params:
    ca_path: "/opt/gitlab/embedded/ssl/certs/"
    ca_file: "/etc/gitlab/ssl/redis-bundle.crt"
    cert_file: "/etc/gitlab/ssl/redis-client.crt"
    key_file: "/etc/gitlab/ssl/redis-client.key"
  sentinels:
    -
      host: localhost
      port: 26379
      ssl: true
      ssl_params:
        ca_file: "/etc/gitlab/ssl/sentinel-bundle.crt"
        cert: "/etc/gitlab/ssl/redis-client.crt"
        key: "/etc/gitlab/ssl/redis-client.key"
    -
      host: localhost
      port: 26380
      ssl: true
      ssl_params:
        ca_file: "/etc/gitlab/ssl/sentinel-bundle.crt"
        cert: "/etc/gitlab/ssl/redis-client.crt"
        key: "/etc/gitlab/ssl/redis-client.key"
$ cat /var/opt/gitlab/gitlab-rails/etc/cable.yml
production:
  adapter: redis
  url: rediss://mymaster/
  ssl: true
  ssl_params:
    ca_path: "/opt/gitlab/embedded/ssl/certs/"
    ca_file: "/etc/gitlab/ssl/redis-bundle.crt"
    cert_file: "/etc/gitlab/ssl/redis-client.crt"
    key_file: "/etc/gitlab/ssl/redis-client.key"
  sentinels:
    -
      host: localhost
      port: 26379
      ssl: true
      ssl_params:
        ca_file: "/etc/gitlab/ssl/sentinel-bundle.crt"
        cert: "/etc/gitlab/ssl/redis-client.crt"
        key: "/etc/gitlab/ssl/redis-client.key"
    -
      host: localhost
      port: 26380
      ssl: true
      ssl_params:
        ca_file: "/etc/gitlab/ssl/sentinel-bundle.crt"
        cert: "/etc/gitlab/ssl/redis-client.crt"
        key: "/etc/gitlab/ssl/redis-client.key"
$ cat /var/opt/gitlab/gitlab-workhorse/config.toml


[redis]
Password = ""
Sentinel = ["rediss://localhost:26379","rediss://localhost:26380"]
SentinelMaster = "mymaster"
SentinelPassword = ""

[Sentinel.tls]
  certificate = "/etc/gitlab/ssl/redis-client.crt"
  key = "/etc/gitlab/ssl/redis-client.key"
  ca_certificate = "/etc/gitlab/ssl/sentinel-bundle.crt"
[redis.tls]
  certificate = "/etc/gitlab/ssl/redis-client.crt"
  key = "/etc/gitlab/ssl/redis-client.key"
  ca_certificate = "/etc/gitlab/ssl/redis-bundle.crt"

[object_storage]
  provider = "AWS"
[object_storage.s3]
  aws_access_key_id = ""
  aws_secret_access_key = ""


[image_resizer]
  max_scaler_procs = 4
  max_filesize = 250000

$ sudo cat /var/opt/gitlab/gitlab-kas/gitlab-kas-config.yml
agent:
  listen:
    address: localhost:8150
    network: tcp
    websocket: true
  configuration:
    poll_period: 300s
  kubernetes_api:
    listen:
      address: localhost:8154
    url_path_prefix: /
    websocket_token_secret_file: /var/opt/gitlab/gitlab-kas/websocket_token_secret_file
  info_cache_ttl: 300s
  info_cache_error_ttl: 60s
gitlab:
  address: https://stanhu-aws.stanhu.com
  external_url: https://stanhu-aws.stanhu.com
  authentication_secret_file: /var/opt/gitlab/gitlab-kas/authentication_secret_file
observability:
  listen:
    address: localhost:8151
    network: tcp
  usage_reporting_period: 60s
  logging:
    level: info
    grpc_level: error
redis:
  network: tcp
  tls:
    enabled: true
    ca_certificate_file: "/etc/gitlab/ssl/redis-bundle.crt"
    certificate_file: "/etc/gitlab/ssl/redis-client.crt"
    key_file: "/etc/gitlab/ssl/redis-client.key"
  sentinel:
    master_name: mymaster
    addresses:
      - 'localhost:26379'
      - 'localhost:26380'
    tls:
      enabled: true
      ca_certificate_file: "/etc/gitlab/ssl/sentinel-bundle.crt"
      certificate_file: "/etc/gitlab/ssl/redis-client.crt"
      key_file: "/etc/gitlab/ssl/redis-client.key"
api:
  listen:
    network: tcp
    address: localhost:8153
    authentication_secret_file: /var/opt/gitlab/gitlab-kas/authentication_secret_file
private_api:
  listen:
    network: tcp
    address: localhost:8155
    authentication_secret_file: /var/opt/gitlab/gitlab-kas/private_api_authentication_secret_file

workspaces:
  enabled: false
config:
  1. Use GitLab and GitLab CI. Monitor gitlab-ctl tail for any Redis errors. NOTE: There is an existing problem with redis-exporter when Sentinels are enabled where the exporter doesn't get the correct address in the --redis.addr host:
$ cat /opt/gitlab/sv/redis-exporter/run
#!/bin/sh
exec 2>&1

umask 077
exec chpst -P -e /opt/gitlab/etc/redis-exporter/env \
  -U gitlab-redis:git \
  -u gitlab-redis:git \
  /opt/gitlab/embedded/bin/redis_exporter --web.listen-address=localhost:9121 --redis.addr=rediss:/

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled workflowready for review per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes.
  • Documentation created/updated.
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened.
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Jason Plum

Merge request reports