Group-level Kubernetes cluster configuration
Problem to solve
Currently, k8s clusters can only be configured at the project level. This can be limiting for teams that want to use the same cluster across all projects/department/company.
A single cluster configuration that can be leveraged group-wide.
Add a new option for "Operations" at the group level that will allow users to access a kubernetes menu item.
The following have been discovered as part of https://gitlab.com/gitlab-org/gitlab-ee/issues/7412:
- Only the following group-level applications will be available for MVC:
- The following applications will be available post-mvc:
- Runner https://gitlab.com/gitlab-org/gitlab-ce/issues/51988
- Prometheus https://gitlab.com/gitlab-org/gitlab-ce/issues/51963
- JupyterHub https://gitlab.com/gitlab-org/gitlab-ce/issues/51989
Group-level cluster will follow the same licensing approach as project-level clusters: CE will allow for 1 cluster at the group level (or sub-group level) and EE will allow for multiple cluster configuration.
For CE, having a cluster configured at the group level shall not affect the ability for related project to have project-level clusters setup, that is, if Group 1 has a cluster setup, project 1A can have a separate cluster setup (project 1A belonging to Group 1).
Adding a group-level cluster will automatically show that cluster to all children projects
Environment scope for CE will enforce
*just as CE project-level cluster configuration does
Project-level cluster trumps group-level cluster, that is, if there's an env scope match, project level always wins.
Sub-groups will determine which cluster to use by looking at closest ancestor with a matching env scope pattern.
- Remove the project namespace from group-level clusters completely
- Don’t show it in the list when listing clusters, either at the group or project level
- Don’t have the field when creating a group-level cluster
- Don’t have it when editing a group-level cluster
- But keep it when creating/editing a project-level cluster as there’s still reason to keep it
- Even though project-level clusters could have custom project namespaces, don't show them in the list of clusters for consistency
- The backend still needs to provide
KUBE_NAMESPACEto CI/CD pipelines for group-level clusters. I suggest keeping the
Managing the cluster can only be done by group maintainers/owners
Removing cluster at the group level will disable the access for its children projects
Discovery issue https://gitlab.com/gitlab-org/gitlab-ee/issues/7412
- Auto DevOps domain will be moved away from project configuration to cluster configuration, to be shown (possibly) along ingress information https://gitlab.com/gitlab-org/gitlab-ce/issues/52363
- Support installing GitLab runner on group level cluster https://gitlab.com/gitlab-org/gitlab-ce/issues/51988
- Show errors if project namespace/SA is missing https://gitlab.com/gitlab-org/gitlab-ce/issues/54506
User stories and mockups
Sidebar and empty state
|As an owner or maintainer, I should see a
|As an owner or maintainer, I should see an empty state with a call to action when I have no group clusters.|
|As an owner or maintainer, I can set the environment scope when creating my group cluster.||As an owner or maintainer, I cannot set the environment scope when creating my group cluster. The environment scope is
Applications and domain
|As an owner or maintainer, I am able to install Helm Tiller and Ingress on my group level cluster. Note: The only difference between EE/CE mockups would be the ability to set an environment.|
|As an owner or maintainer, I am able to set a domain for both my group and project level clusters. I am able to set my domain regardless of whether I have installed Helm Tiller and/or Ingress.|
Improvements: Alternative issues will be made for installing GitLab Runner, Prometheus, and Jupyterhub &114 (closed)
|As an owner or maintainer, I am able to add multiple group clusters to each group or subgroup.||As an owner or maintainer, I can only add one group cluster to each group or subgroup.|
|Add cluster button enabled||Add cluster button disabled|
|Group level clusters will be automatically added to projects within that group.|
|As a user who created a new group cluster, I will be warned if my cluster won't be active for any given project.|
Improvement: Allow users to specify which projects they would like override. Create new issue
|As an owner or maintainer, I can have multiple active cluster integrations at various levels on my project.||As an owner or maintainer, I can only have one active cluster integration on my project.|
|As an owner or maintainer, I have the ability to add one project level cluster even if there are group clusters. This will override my group level integration for this project.|
|Add cluster button enabled|
Possible improvement: Explore always keeping the create button enabled. In CE, this would mean owners and maintainers can add as many integrations as they want, but they must select one: either a group or a project level cluster. This removes confusion over why the user cannot create another integration. Create new issue
Editing group clusters
Improvement: Allow users to exclude a group level cluster from a specific project. Create new issue
Auto DevOps project settings
|As an owner or maintainer, I am no longer able to set a domain within my project Auto DevOps settings|
What does success look like, and how can we measure that?
User no longer copy/paste the same cluster configuration in multiple projects but rather use a group-level cluster
Links / references
/label ~"feature proposal"