Skip to content
Snippets Groups Projects

Initial web terminal design doc

Merged Andrew Fontaine requested to merge afontaine/web-terminal-design-doc into master
All threads resolved!

We want to provide users the ability to manipulate their kubernetes clusters from within GitLab. This is the initial design for it.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Timo Furrer
  • Timo Furrer
  • @afontaine thanks for writing this up! I've left a bunch of comments :smile:

  • Timo Furrer requested review from @timofurrer

    requested review from @timofurrer

  • Andrew Fontaine added 1 commit

    added 1 commit

    • a2090020 - Refine web terminal design doc

    Compare with previous version

  • Andrew Fontaine added 1 commit

    added 1 commit

    • 61ae40d6 - Add bit on user impersonation, add KAS to diagram

    Compare with previous version

    • Resolved by Andrew Fontaine

      @afontaine While considering how GitLab-Rails should manage the pods, I wonder if we can take a bit simpler approach without using GitLab-Rails at all for the initial iteration:

      • GitLab-Frontend communicates with customer's Kubernetes with the user_access keyword. This already exists.
      • When user clicks "Web Terminal" button, GitLab-Frontend fetches currently running ephemeral pods as a pre-flight check. If the count exceeds 5, we show an error message like "There are too many ephemeral pods, try again later or terminate others at first".
      • If the pre-flight check passes, GitLab-Frontend creates an ephemeral pod which will expire in 10 minutes. We can add the TTL logic inside the container image.
      • GitLab-Frontend attaches to the pod.

      A tricky part is what kube config we'll inject inside the pod. Currently, authorizations and RBAC are mainly handled in the KAS, but it seems we're proposing that the pod should directly communicate with the cluster for reducing latency. We need to carefully implement it otherwise it could be a security hole that users can escalate access from an impersonated SA to admin. My gut tells me that we should always go through KAS even if it sacrifice the latency given that the security would be the most important part and cluster-debugging wouldn't happen often.

  • Timo Furrer removed review request for @timofurrer

    removed review request for @timofurrer

  • Pam Artiaga
  • Andrew Fontaine added 1 commit

    added 1 commit

    • cb7d8182 - Expand SA to service account

    Compare with previous version

  • requested review from @partiaga, @ash2k, and @shinya.maeda

  • Shinya Maeda removed review request for @shinya.maeda

    removed review request for @shinya.maeda

    • Resolved by Andrew Fontaine

      @afontaine I was thinking about user impersonation and I think we have two options:

      • What is currently documented: using PAT+making the pod go via kas rather than direct kube API access.
      • Supporting doing impersonation locally in agentk: make pod access kubernetes API via agentk. I.e. agentk would be a reverse proxy for the terminal pods. We'd inject some sort of credential into kubeconfig that agentk could validate - completely locally or by asking kas+caching. We could even use a PAT as that credential (with validation in kas).

      Could you mention the second option in the proposal please? Just so that we don't lose track of the idea.

      @nagyv-gitlab I actually think this may be an interesting product feature. Allow user to access their cluster using a PAT but locally, not via kas. If they change the API of the cluster to point at agentk's dedicated port, use PAT, they get user impersonation + low latency access + RBAC control based on user groups. Might be a good GitLab Premium or GitLab Ultimate feature. It's a variation on what we have almost finished building with glab creating a context + PAT in kubeconfig.

      edit: hm, agentk would have to use TLS and this might be a bit tricky, but we could generate self-signed certs and trust them via the kubeconfig, this is not hard.

      Edited by Mikhail Mazurskiy
  • Andrew Fontaine added 1 commit

    added 1 commit

    • 83fc272e - Add note about local user impersonation in agentk

    Compare with previous version

    • Resolved by Mikhail Mazurskiy

      Thanks for the additional feedback @ash2k.

      @ash2k @shinya.maeda @partiaga we seem to be stuck at a point on how to clean up old web terminal pods. We can spin this out to a further iteration, but I'd like to take a couple more days to get a rough idea into a doc we can iterate on later.

      As it is, I am exhausting my backend knowledge regarding secure auth, so aside from somehow saving a token for later or forcing users to click a Clean up dead sessions button, I am not sure what a good approach would be.

  • Andrew Fontaine added 1 commit

    added 1 commit

    • b378b29a - Move deployment/cleanup to KAS

    Compare with previous version

  • Andrew Fontaine added 1 commit

    added 1 commit

    • c76b9c39 - Clarify who keeps track of current web terminals

    Compare with previous version

  • Mikhail Mazurskiy resolved all threads

    resolved all threads

  • Mikhail Mazurskiy approved this merge request

    approved this merge request

  • Mikhail Mazurskiy started a merge train

    started a merge train

  • mentioned in commit 3232982a

  • added workflowcanary label and removed workflowproduction label

  • Please register or sign in to reply
    Loading