Initial web terminal design doc
We want to provide users the ability to manipulate their kubernetes clusters from within GitLab. This is the initial design for it.
Merge request reports
Activity
added groupenvironments label
added devopsdeploy sectioncd labels
1 Warning Please add a merge request subtype to this merge request. Reviewer roulette
Changes that require review have been detected! A merge request is normally reviewed by both a reviewer and a maintainer in its primary category and by a maintainer in all other categories.
To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, mention them as you normally would! Danger does not automatically notify them for you.
Reviewer Maintainer No reviewer available @hfyngvason
(UTC-4, same timezone as
@afontaine
)If needed, you can retry the
danger-review
job that generated this comment.Generated by
Dangeradded design-doc label
added docs-only label
- Resolved by Andrew Fontaine
@afontaine - please see the following guidance and update this merge request.1 Error Please add typebug typefeature, or typemaintenance label to this merge request.
Hey @ash2k, @timofurrer I have attempted to distill the designs we've been discussing into this document.
I think I've captured everything here, but I imagine this is lacking in some details, and I'd be grateful for the assistance in fleshing it out, specifically with regards to what GitLab needs and what the Agent needs when it comes to requests, etc.
I am also not entirely sure if there's other work needed to implement the design, please chime in here as well.
@gitlab-org/ci-cd/deploy-stage/environments-group please also chime in as well with any thoughts
added typemaintenance label
assigned to @afontaine
- Resolved by Shinya Maeda
Thanks @afontaine for writing up! This is interesting proposal.
What problem does this feature resolve where PAT access and
glab
CLI support can't resolve? In my understanding, with Access a cluster with the Kubernetes API feature, users can basically execute any tools such askubectl
,helm
, etc (some commands likeexec
orattach
might not supported yet but could be a follow-up).So it seems the problem we're trying to solve here is skipping the CLI binary installation like
kubectl
, however, it would be relatively easy for the target audiences as it just downloads the binary into their local computer. What would be the actual value of "Web" terminal instead of local terminal?
- Resolved by Shinya Maeda
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Mikhail Mazurskiy
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
@afontaine thanks for writing this up! I've left a bunch of comments
requested review from @timofurrer
- Resolved by Andrew Fontaine
- Resolved by Mikhail Mazurskiy
- Resolved by Andrew Fontaine
added 1 commit
- 61ae40d6 - Add bit on user impersonation, add KAS to diagram
- Resolved by Andrew Fontaine
@afontaine While considering how GitLab-Rails should manage the pods, I wonder if we can take a bit simpler approach without using GitLab-Rails at all for the initial iteration:
- GitLab-Frontend communicates with customer's Kubernetes with the
user_access
keyword. This already exists. - When user clicks "Web Terminal" button, GitLab-Frontend fetches currently running ephemeral pods as a pre-flight check. If the count exceeds 5, we show an error message like "There are too many ephemeral pods, try again later or terminate others at first".
- If the pre-flight check passes, GitLab-Frontend creates an ephemeral pod which will expire in 10 minutes. We can add the TTL logic inside the container image.
- GitLab-Frontend attaches to the pod.
A tricky part is what kube config we'll inject inside the pod. Currently, authorizations and RBAC are mainly handled in the KAS, but it seems we're proposing that the pod should directly communicate with the cluster for reducing latency. We need to carefully implement it otherwise it could be a security hole that users can escalate access from an impersonated SA to admin. My gut tells me that we should always go through KAS even if it sacrifice the latency given that the security would be the most important part and cluster-debugging wouldn't happen often.
- GitLab-Frontend communicates with customer's Kubernetes with the
removed review request for @timofurrer
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
- Resolved by Andrew Fontaine
@ash2k @shinya.maeda @partiaga I want to get this doc in ship-shape and merged. We have a few outstanding discussions looking for feedback:
I'd like to resolve these before we merge, but we can iterate on the clean-up process going forward.
Let's do one more round of review, and then tidy things up for merging by the end of next week, Oct 20 (as I am OOO for the rest of this week)
requested review from @partiaga, @ash2k, and @shinya.maeda
removed review request for @shinya.maeda
- Resolved by Andrew Fontaine
@afontaine I was thinking about user impersonation and I think we have two options:
- What is currently documented: using PAT+making the pod go via kas rather than direct kube API access.
- Supporting doing impersonation locally in agentk: make pod access kubernetes API via agentk. I.e. agentk would be a reverse proxy for the terminal pods. We'd inject some sort of credential into kubeconfig that agentk could validate - completely locally or by asking kas+caching. We could even use a PAT as that credential (with validation in kas).
Could you mention the second option in the proposal please? Just so that we don't lose track of the idea.
@nagyv-gitlab I actually think this may be an interesting product feature. Allow user to access their cluster using a PAT but locally, not via kas. If they change the API of the cluster to point at agentk's dedicated port, use PAT, they get user impersonation + low latency access + RBAC control based on user groups. Might be a good GitLab Premium or GitLab Ultimate feature. It's a variation on what we have almost finished building with glab creating a context + PAT in kubeconfig.
edit: hm, agentk would have to use TLS and this might be a bit tricky, but we could generate self-signed certs and trust them via the kubeconfig, this is not hard.
Edited by Mikhail Mazurskiy
added 1 commit
- 83fc272e - Add note about local user impersonation in agentk
- Resolved by Mikhail Mazurskiy
Thanks for the additional feedback @ash2k.
@ash2k @shinya.maeda @partiaga we seem to be stuck at a point on how to clean up old web terminal pods. We can spin this out to a further iteration, but I'd like to take a couple more days to get a rough idea into a doc we can iterate on later.
As it is, I am exhausting my backend knowledge regarding secure auth, so aside from somehow saving a token for later or forcing users to click a
Clean up dead sessions
button, I am not sure what a good approach would be.
mentioned in issue gitlab-org/gitlab#418264
added 1 commit
- c76b9c39 - Clarify who keeps track of current web terminals
- Resolved by Mikhail Mazurskiy
mentioned in commit 3232982a
added workflowproduction label
added workflowcanary label and removed workflowproduction label
added workflowstaging-canary label and removed workflowcanary label
added workflowstaging label and removed workflowstaging-canary label
added releasedcandidate label
added releasedpublished label and removed releasedcandidate label