Skip to content

Product Discovery for Group-Level K8S Cluster Configuration

Background:

Currently K8S clusters can only be configured at the project level. If the same cluster is being used for multiple projects, it must be manually added multiple times.

We want provide users the ability to do Group-level K8S cluster config so a single cluster can be shared across projects.

What questions are you trying to answer?

What is the smallest sequence of mergeable steps that can be done for this?

See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21831#possible-mrs (⚠ WIP)

Are you looking to verify an existing hypothesis or uncover new issues you should be exploring?

No existing hypothesis

What is the backstory of this project and how does it impact the approach?

GitLab has provided the ability to connect a K8S cluster since 10.1, however, it has always taken place at the project level. This does not allow organization that use clusters at a group or org level to define a cluster once and use it across the entire org. We want the user to be able to easily define at which level they want to configure this cluster.

What do you already know about the areas you are exploring?

What does success look like at the end of the project?

Technical discovery for this issue should inform the MVC for the implementation issue.

Decisions

  • We will only install group-level cluster applications in group clusters - https://gitlab.com/gitlab-org/gitlab-ce/issues/48418#note_103788885 (projects install cluster apps into group clusters should be a new issue)
  • We will ship without Prometheus in the first release - https://gitlab.com/gitlab-org/gitlab-ee/issues/7412#note_103921412
  • We will ship with nginx-ingress configured to watch cluster-wide. (no change from project-level nginx-ingress) - https://gitlab.com/gitlab-org/gitlab-ce/issues/48418#note_103773744
  • We will ship without Jupytyerhub and Runner in the first release
  • Group-level cluster will follow the same licensing approach as project-level clusters: CE will allow for 1 cluster at the group level (or sub-group level) and EE will allow for multiple cluster configuration (first iteration is likely to include single cluster capability). For example, on CE, Group 1 and sub-group A can each have 1 cluster only, on EE multiple cluster are supported.
  • For CE, having a cluster configured at the group level shall not affect the ability for related project to have project-level clusters setup, that is, if Group 1 has a cluster setup, project 1A can have a separate cluster setup (project 1A belonging to Group 1).
  • Group cluster would be available to projects under their namespace. - https://gitlab.com/gitlab-org/gitlab-ee/issues/7412#note_104034851

User stories and mockups

Sidebar and empty state

EE/CE
As an owner or maintainer, I should see a Kubernetes tab in my group sidebar.
As an owner or maintainer, I should see an empty state with a call to action when I have no group clusters.
group__operations--kubernetes-empty-state

Environment scope

EE CE
As an owner or maintainer, I can set the environment scope when creating my group cluster. As an owner or maintainer, I cannot set the environment scope when creating my group cluster. The environment scope is *.
EE-group__operations--kubernetes-create CE-group__operations--kubernetes-create

Applications and domain

EE/CE
As an owner or maintainer, I am able to install Helm Tiller and Ingress on my group level cluster. Note: The only difference between EE/CE mockups would be the ability to set an environment.
As an owner or maintainer, I am able to set a domain for both my group and project level clusters.
Group Project
CE-group__operations--kubernetes-domain-applications Helm and Ingress not installed, domain disabled CE-project__operations--kubernetes-domain-applications Helm and Ingress not installed, domain disabled
CE-group__operations--kubernetes-domain-applications-installed Helm and Ingress installed, domain enabled CE-project__operations--kubernetes-domain-applications-installed Helm and Ingress installed, domain enabled

Improvements: Alternative issues will be made for installing GitLab Runner, Prometheus, and Jupyterhub &114 (closed)

Group clusters

EE CE
As an owner or maintainer, I am able to add multiple group clusters to each group or subgroup. As an owner or maintainer, I can only add one group cluster to each group or subgroup.
EE-group__operations--kubernetes-one-project Add cluster button enabled CE-group__operations--kubernetes-one-project Add cluster button disabled
EE/CE
Group level clusters will be automatically added to projects within that group.

Cluster Overrides

CE
As a user who created a new group cluster, I will be warned if my cluster won't be active for any given project.
CE-group__operations--kubernetes-warning

Improvement: Allow users to specify which projects they would like override. Create new issue

EE CE
As an owner or maintainer, I can have multiple active cluster integrations at various levels on my project. As an owner or maintainer, I can only have one active cluster integration on my project.
EE-project__operations--kubernetes-two CE-project__operations--kubernetes-two
EE-project__operations--kubernetes-four-2 CE-project__operations--kubernetes-four
EE
If multiple clusters exist at the same level, my environment will first look for an exact match, then a partial match, then *
If there are multiple matches, my environment will choose the first match. Project -> Subgroup -> Group
EE-project__operations--kubernetes-four-1
If my environment has no match, it will continue up the chain to find a match. Project -> Subgroup -> Group
CE
As an owner or maintainer, I have the ability to add one project level cluster even if there are group clusters. This will override my group level integration for this project.
CE-project__operations--kubernetes-one-group Add cluster button enabled
CE-project__operations--kubernetes-override-integration

Possible improvement: Explore always keeping the create button enabled. In CE, this would mean owners and maintainers can add as many integrations as they want, but they must select one: either a group or a project level cluster. This removes confusion over why the user cannot create another integration. Create new issue

Editing group clusters

EE/CE
As an owner or maintainer, I can view a group cluster on my project. The cluster links to the group cluster page, and I must make any edits there.
As an owner or maintainer, I cannot remove a group level cluster from a specific project. I can only override the cluster with a project cluster.
CE-project__operations--kubernetes-one-group

Improvement: Allow users to exclude a group level cluster from a specific project. Create new issue

Auto DevOps project settings

EE/CE
As an owner or maintainer, I am no longer able to set a domain within my project Auto DevOps settings
CE-project__settings--ci-cd-autodevops

Improvement: Resolve any UX Debt around deployment strategy, especially after moving the domain to cluster settings. Create new issue

Links / references:

Implementation issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/34758

Edited by Taurie Davis