Use the agent as a reverse tunnel to connect to a cluster directly from CI
Release notes
Problem to solve
As an Application Operator, I would like to use the GitLab Kubernetes Agent for deployments as part of my CI job, so that I don't have to open up my cluster to connect it to GitLab.
Intended users
User experience goal
The user can configure their CI to send kubernetes manifests over for one or more agents to deploy.
We want to support a single pipeline to deploy to multiple clusters, that is to deploy using multiple Agents. This is often required in production environments, where the production environment is composed of multiple clusters in different regions/availability zones. This would be possible using matrix builds.
Proposal
Enable the Runner job to talk directly through kas to a cluster on private network, so that commands such as kubectl
can be run directly against the cluster.
Initially, we can provide the KUBECONTEXT
with the agent's access rights, thus we would gain a similar experience to the non-gitlab managed apps today: the CI would be allowed to do anything that the agent has rights to do. We might change this once #243740 (closed) is implemented.
Iteration 1
In the first version, we are aiming for single-project setups. That is the CI can access the agent(s) registered in the same project. Moreover, this behaviour is enabled by default, and is available to any CI job. This can be the equivalent of the current project level cluster integrations, except for managing namespaces.
Initial metric: increment a metric +1 per agent when a request hits kas.gitlab.example.com and report it in usage ping.
- How to make sure that the deployment token is still alive? The
CI_JOB_TOKEN
becomes invalid once the job is over.
Further iteration are still being discussed.
Iteration 2
Allow for group and instance level agents (might mean 2 separate iterations). A group level agent means that any project within that group can use the Agent as a deployment target.
- Do we expect the Agent to be in that group? No.
- Do we allow the same agent to be the group level agent of many groups? Yes.
Questions
- How do we select which agent to use for the deployment when the project has multiple agent's configured?
- It's up to the user. Using a matrix pipeline, it might use multiple agents too.
- Can the CI job use an agent outside of the given project?
- Not in the first iteration
- How can the CI job use a group level agent?
- Not in the first iteration
- How are CI projects authorized to use an agent as a tunnel?
- In the first iteration we start with a single-project setup where the project should contain the agent
- Shall we use the
environment
keyword to specify the agent or introduce a new one? - How can we measure usage?
Further details
Use cases
Given a classical project that deploys to k8s
And the project containing a registered agent with name `my-agent-name`
When the CI contains
my_job:
...
environment:
name: my-agent-name
Then the given job receives a `$KUBECONTEXT` to access the cluster of `my-agent-name` agent
Given a classical project that deploys to k8s in a group
And a group having a group level registered agent with name `my-group-agent`
When the CI contains
my_job:
...
environment:
name: my-group-agent
Then the given job receives a `$KUBECONTEXT` to access the cluster of `my-agent-name` agent
Permissions and Security
- The user running the CI job should have Developer rights on the project configuring the agent used.
- This feature can be disabled in the agent configuration - iteration 2
Documentation
Availability & Testing
What does success look like, and how can we measure that?
We expect this feature to increase the number of Agents being installed.
The related metrics are still being discussed:
What is the type of buyer?
Pick one for the tier: