# Problem to solve As a Software Developer, I want insights into a cluster around the status of my deployments, tailed logs and overall metrics, so I can troubleshoot a deployment from a review environment to production. As an Application Operator, I want to follow how a deployment is being rolled out / rolled back, see the related metrics and be warned if my intervention is required, so I can deploy often and with confidence. ## Current situation Today GitLab offers [Environment views](https://docs.gitlab.com/ee/ci/environments/#view-environments-and-deployments). These views provide the action to - redeploy the last deployment - stop the environment - deploy a manual job to the environment These views when the project has a certificate-integrated Kubernetes cluster attached show - the running pods with their status - have shortcuts to the related metrics - open a web terminal Some of these features are considered unusable (web terminal) or confusing (deploying a manual job) in their current form by our users. The cluster insights are considered to be really minimal that can be used only as a very first step to check the status of pods during and after a deployment. ## Requirements Our users ask for environment insights, including (by priority order): 1. **what is currently deployed into the environment and their status** 2. **log tailing for the deployed applications** 2. metrics for the deployed applications 3. log search for the deployed appplications 4. console access to pods (pod terminal) The scope here is primarily the first two items above, but it might be useful to think about the broader context to see the long term goal. We want to provide these features, here focusing specifically to Kubernetes and the "what is currently deployed into the environment" question with integration points to logs and metrics. The information should be provided about any resource available given the account connection and the level of view used. This account might be the service account used by `agentk` or a user-impersonated account. These views should be near real-time updated to provide up to date information about changing resources as well, like scaling out a deployment, or tailing logs. Beside showing the pods, the user should be able to access other managed resources as well, like configmaps, etc what they have rights to access given the used account. ### Information shown Let's assume the following setup: The user has 3 agents under a group and accordingly 2 environments (dev, prod): - dev - prod-us - prod-eu There are multiple projects under this group, each deploying with the mentioned 3 connections, there is another project outside the group using these agents too. At the group level, we should show the 2 environments and all the resources deployed by all the projects under the group, but not the resources deployed by the project outside the group. At each project level, we should show the 2 environments and all the resources deployed by that single project only. In all cases, the resources shown should be restricted to the actual account used to retrieve them, using Kubernetes RBAC. In ~"GitLab Core", we can support the agents' service account, while in ~"GitLab Premium" the platform engineer can configure to use GitLab user impersonation, and further restrict the available resources. (For example, secrets can be read only by project maintainers.) A kubernetes cluster consists of static and dynamic information as well. For example a Deployment or Configmap resource is static, while pods are dynamic. For static resources, its enough to provide a description view, so the user can check a configuration value for example. For dynamic resources, beside providing the description view, we should provide dynamic views as well, like updating the status of the resource, providing log tailing, console access, etc ... ### Technical aspect The pull based deployments work through an inventory object. An inventory object defines a common annotation that is set on all the managed resources. Filtering by annotations is not supported by kubernetes, so we would need to retrieve all the resources and filter the annotations by code. Pull based deployments by default don't have any mechanism to describe where they originate from. Usually labels or annotation are applied. It might be possible to apply annotations/labels that tie the resources to the group/project/job that created them using an admission controller. The UI uses labels to find the pods to show. Some of our users use these labels without using our deployment integrations. ## Proposal Leverage the Kubernetes Agent for the connection between GitLab (UI) and the cluster. Build on an existing kubernetes dashboard and contribute back to the community where appropriate. The Kubernetes spec proposes [the Application CRD](https://github.com/kubernetes-sigs/application/blob/master/docs/api.md) to describe a group of related resources. Do we want to add special support to it? Like automatically adding an Application resource together with every project / pull deployment? ### Iterations - Integrate an existing UI similar to the [GraphiQL integration](https://gitlab.com/-/graphql-explorer) - Link to the UI either from the agent detail page or from the group level - Extend the UI with the top level menu and a back to GitLab button similar to the [Web IDE](https://gitlab.com/-/ide/project/gitlab-org/gitlab/edit/master/-/) ## Not in scope There are many tools that provide full cluster management from the UI, including installing new applications/operators, editing resource yaml, etc. This is outside the scope of this epic. Our approach is to use `git` as the single source of truth to describe what is installed in the cluster and what resources are managed through GitLab. As a result, editing yaml would require integrations with GitLab, and similarly, we would prefer to have application installations be registered in a `git` repository too. ## Other solutions - ArgoCD has a highly valued UI - [k9s](https://github.com/derailed/k9s) - [official kubernetes dashboard](https://github.com/kubernetes/dashboard) - [k8s lens](https://k8slens.dev/) - Kubernetic - Infra.app - Kubenav - Kuber - https://github.com/vmware-tanzu/octant - https://github.com/indeedeng/k8dash - https://github.com/benc-uk/kubeview - https://octopus.com/blog/alternative-kubernetes-dashboards - [KUI](https://github.com/kubernetes-sigs/kui) is [largely customizable](https://github.com/kubernetes-sigs/kui/wiki/5.-How-to-Customize-Your-Client)