Skip to content

GitLab Next

GitLab FOSS
GitLab FOSS
You need to sign in or sign up before continuing.
Closed
Opened by Fabio Busatto@bikebillyContributor

Instance level Kubernetes cluster configuration

Description

Kubernetes clusters are now configured in CI/CD > Cluster page, and eventually the old service integration page will be removed (https://gitlab.com/gitlab-org/gitlab-ce/issues/39217).

When it will happen, there will be no way to define a cluster at instance level, because service templates will not cover it anymore. We should find a way to define clusters at instance level, to allow projects on on-prem installations to get a configuration.

It is partially covered by group-level cluster page (https://gitlab.com/gitlab-org/gitlab-ce/issues/34758), but if a company doesn't work with groups, or has a lot of them, an instance-wide configuration can be better.

Proposal

  • Create a cluster configuration at the instance level, which replaces the Kubernetes service template.
  • Follows same model as project/group clusters where CE license can have 1 cluster and EE license can have multiple clusters
  • Deprecate Kuberenetes service template
  • Creation of Kubernetes resources for projects using CI when the pipeline is run
  • Error handling and surfacing errors for the creation of Kubernetes resources will use the current build errors we already have
  • The rest of the experience will mimic group level clusters
  • Cluster precedence will follow the model implemented for group-level clusters, where precedence is: project >> sub-group >> group >> instance
  • initially helm and ingress available
  • prometheus issue https://gitlab.com/gitlab-org/gitlab-ce/issues/52995
  • runner issue https://gitlab.com/gitlab-org/gitlab-ce/issues/53076
  • update usage.rb to account for instance-level clusters (similar to group-level https://gitlab.com/gitlab-org/gitlab-ee/blob/master/lib/gitlab/usage_data.rb#L63)
Empty state
admin__operations--kubernetes-empty-state
CE Create EE Create
CE-admin__operations--kubernetes-create EE-admin__operations--kubernetes-create

Links / references

Design artifact: https://gitlab.com/gitlab-org/gitlab-ce/issues/48649

Conclusions from product discovery https://gitlab.com/gitlab-org/gitlab-ce/issues/48649:

  1. Defer namespace creation until project has a deployment CI job. This means projects with only build and test jobs will not create namespaces. Only projects which deploy to the cluster will have namespaces and accounts created (this also provides the env name which is required to create k8s resources in the right place). https://gitlab.com/gitlab-org/gitlab-ce/issues/57115
  2. Error handling and surfacing errors for creation of these Kubernetes resources can use the current build errors
  3. On project delete, automatically delete namespace
  4. On project rename, persist the project namespace name (<project_name>-<project_id>). It will remain unchanged in DB so we will continue to use the same namespace even though project is renamed
  5. Cleaning up namespaces for deleted projects will be done in a follow-up issue https://gitlab.com/gitlab-org/gitlab-ce/issues/53591
Edited by Daniel Gruesso