Add TLS configuration to Consul

What does this MR do?

This MR adds attributes for configuring TLS on Consul cluster nodes. By default TLS is not enabled for Consul cluster. You can use the following attributes to enable it. They work for both client and server nodes.

consul['use_tls'] = false
consul['tls_ca_file'] = nil
consul['tls_certificate_file'] = nil
consul['tls_key_file'] = nil 
consul['tls_verify_client'] = nil

In order to support these settings, this MR also pull out HTTP/S ports from consul['configuration'] and introduces them as top-level attributes:

consul['http_port'] = nil
consul['https_port'] = nil

How it works

To enable TLS on a Consul node start with consul['use_tls'] = true. Depending on the role of the node (server or client) and your TLS preferences you need to provide further configuration. For example, on a server node you must at least specify tls_ca_file, tls_certificate_file, and tls_key_file. On a client node, when client TLS authentication is disabled on server (it is enabled by default) you must at least specify tls_ca_file otherwise you will have to pass the client TLS certificate and key via tls_certificate_file, and tls_key_file.

When TLS is enabled, by default the server uses mTLS and listens on both HTTPS and HTTP (as wells as TLS and non-TLS RPC). It expects clients to use TLS authentication. You can disable client TLS authentication by setting consul['tls_verify_client'] = false.

On the other hand, clients only use TLS for outgoing connection to server nodes and only listen on HTTP (as well as non-TLS RPC) for incoming requests. You can enforce client Consul agents to use TLS for incoming connections by setting consul['https_port'] to a non-negative integer (8501 is the Consul's default HTTPS port). You must also pass tls_certificate_file and tls_key_file for this to work. When server nodes use client TLS authentication, the client TLS certificate and key is used for both TLS authentication and incoming HTTPS connections.

Consul client nodes do not use TLS client authentication by default (as opposed to servers) and you need to explicitly instruct them to do it by setting consul['tls_verify_client'] = true.

Why it doesn't use auto_encrypt?

When auto_encrypt is used the server automatically distributes the certificates to the clients. This works when the communication between Consul client and server nodes. However it can't be easily configured for other clients that are use TLS client authentication (for example Patroni nodes). That's why the TLS certificates and keys must be generated and distributed manually over different nodes.

Example usage

Minimal TLS support: Server uses TLS for incoming connections (without client TLS authentication)

Server node config:

consul['enable'] = true
consul['configuration'] = {
  'server' => true
}

consul['use_tls'] = true
consul['tls_ca_file'] = '/path/to/ca.crt.pem'
consul['tls_certificate_file'] = '/path/to/server.crt.pem'
consul['tls_key_file'] = '/path/to/server.key.pem'
consul['tls_verify_client'] = false

Client node config (e.g. Patroni node):

consul['enable'] = true

consul['use_tls'] = true
consul['tls_ca_file'] = '/path/to/ca.crt.pem'

patroni['consul']['url'] = 'http://localhost:8500'

Note that Patroni talks to the local Consul agent which does not use TLS for incoming connections. Hence the HTTP URL for patroni['consul']['url'].

Default TLS support: Server uses mutual TLS authentication

Server node config:

consul['enable'] = true
consul['configuration'] = {
  'server' => true
}

consul['use_tls'] = true
consul['tls_ca_file'] = '/path/to/ca.crt.pem'
consul['tls_certificate_file'] = '/path/to/server.crt.pem'
consul['tls_key_file'] = '/path/to/server.key.pem'

Client node config (e.g. Patroni node):

consul['enable'] = true

consul['use_tls'] = true
consul['tls_ca_file'] = '/path/to/ca.crt.pem'
consul['tls_certificate_file'] = '/path/to/client.crt.pem'
consul['tls_key_file'] = '/path/to/client.key.pem'

patroni['consul']['url'] = 'http://localhost:8500'

Note that Patroni talks to the local Consul agent which does not use TLS for incoming connections, even though it uses TLS authentication to Consul server nodes. Hence the HTTP URL for patroni['consul']['url'].

Full TLS support: Both client and server use mutual TLS authentication

Server node config:

consul['enable'] = true
consul['configuration'] = {
  'server' => true
}

consul['use_tls'] = true
consul['tls_ca_file'] = '/path/to/ca.crt.pem'
consul['tls_certificate_file'] = '/path/to/server.crt.pem'
consul['tls_key_file'] = '/path/to/server.key.pem'

Client node config (e.g. Patroni node):

consul['enable'] = true

consul['use_tls'] = true
consul['tls_verify_client'] = true
consul['tls_ca_file'] = '/path/to/ca.crt.pem'
consul['tls_certificate_file'] = '/path/to/client.crt.pem'
consul['tls_key_file'] = '/path/to/client.key.pem'
consul['https_port'] = 8501

patroni['consul']['url'] = 'https://localhost:8501'
patroni['consul']['cacert'] = '/path/to/ca.crt.pem'
patroni['consul']['cert'] = '/opt/tls/patroni.crt.pem'
patroni['consul']['key'] = '/opt/tls/patroni.key.pem'
patroni['consul']['verify'] = true

Note that Consul server, client, and Patroni client certificates must be issued by the same CA for mutual TLS authentication to work.

Related issues

Closes: #6653

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

• Merge Request Title, and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• Pipeline is green on dev.gitlab.org if the change is touching anything besides documentation or internal cookbooks
• trigger-package has a green pipeline running against latest commit

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for the GitLab Chart opened