Improve documentation on KAS certificate settings
Discussion from Slack
I’m reading through the documentation for configuring Kubernetes Agent Server for multi-node installations https://docs.gitlab.com/ee/administration/clusters/kas.html#enable-on-multiple-nodes and have a few questions. I am responsible for the server-side configuration, while our users/teams will be configuring the agent on their respective clusters.
- How do the keys work? gitlab_kas[‘api_secret_key’] and gitlab_kas[‘private_api_secret_key’]. Will I need to share them with each team? The agent install https://docs.gitlab.com/ee/user/clusters/agent/install/index.html#register-the-agent-with-gitlab states to use “this token” -- which token is it referring to?
- Do I simply create these tokens by generating a random 32 character string and then base64 encoding it.
- For gitlab_rails[‘gitlab_kas_internal_url’], it looks like it’s recommending an internal domain that the nodes will use to communicate. For a multi-node configuration, would this mean a new load balancer that only the nodes can use? Or is this for a setup where only a subset of nodes are running the agent server?
- gitlab_kas[‘env’] is suggesting there is a need for SSL certificates here. How and why?
- gitlab_kas_external_url is suggesting using a separate domain for this. Why wouldn’t we just use the main domain?
We are not deprecating this, but the documentation is pretty light on most of these questions.
I asked about the use case for not using Flux and he indicated. - One response: “The main use case would be to deploy kubernetes infrastructure level components to a given cluster, so, even things like keeping gitlab runners up to date in a given set of namespaces.”
How do the keys work? gitlab_kas[‘api_secret_key’] and gitlab_kas[‘private_api_secret_key’]. Will I need to share them with each team?
No need to share them with teams. These keys are responsible for the KAS - GitLab Rails and KAS - KAS communication. The agent installation token referred to in https://docs.gitlab.com/ee/user/clusters/agent/install/index.html#register-the-agent-with-gitlab is a different token needed by agentk to authenticate with KAS. This token will be generated in the project where the agent is registered.
Do I simply create these tokens by generating a random 32 character string and then base64 encoding it.
Yes.
gitlab_kas[‘env’] is suggesting there is a need for SSL certificates here. How and why?
I’ll need engineering support to get the full answer.
Timo could you tell us if these certs are used for KAS-KAS, KAS-Rails or KAS-agentk communication?
What does the how part refer to? How does GitLab use these certificates or how to generate them? I expect the latter to on Disney to answer. We ca help with the former.
gitlab_kas_external_url is suggesting using a separate domain for this. Why wouldn’t we just use the main domain?
We recommend configuring KAS under a subdomain of the main GitLab domain. This allows the widest possible integrations. e.g. gitlab.com -> kas.gitlab.com` Another approach is to use the same domain, in which case KAS would be available under a dedicated path, but I don’t know if this is supported with multi-node installations. (Timo, could you check, please?)
gitlab_kas[‘env’] is suggesting there is a need for SSL certificates here. How and why?
It suggests that you set SSL_CERT_DIR to a directory that contains the SSL certs so that KAS can verify the certs provided to it.
It’s a standard variable used by many things, one of which is the golang std lib:
On Unix systems other than macOS the environment variables SSL_CERT_FILE and SSL_CERT_DIR can be used to override the system default locations for the SSL certificate file and SSL certificate files directory, respectively. The latter can be a colon-separated list.
Thus, it’s for all kinds of communications
Another approach is to use the same domain, in which case KAS would be available under a dedicated path, but I don’t know if this is supported with multi-node installations.
It should be supported. The Enable on multiple nodes docs page lists under it’s Agent server node settings for the gitlab_kas_external_url setting the following note:
The user-facing URL for the in-cluster agentk. Can be a fully qualified domain or subdomain, 1 or a GitLab external URL. 2 If blank, defaults to a GitLab external URL.
with (1) and (2) being:
- For example, wss://kas.gitlab.example.com/.
- For example, wss://gitlab.example.com/-/kubernetes-agent/.
Put differently, SSL_CERT_DIR does not imply that kas expects incoming traffic to be encrypted. It’s about certificates that are used while establishing outbound TLS connections where configured (e.g. to rails, gitaly, kas->kas traffic, Sentry, Google profiler, etc). If kas is not explicitly configured to use TLS for inbound traffic, it will not.