Spike: How to expose Gitlab managed logs in the filtered search
Problem
In the log explorer today a user can filter by 2 parameters, an environment (which is actually a namespace) and by a pod name, while this provides an insight to the app that is deployed in a cluster this view lacking the information of the actual cluster and the gitlab mng apps (e.g. K8s logs and other managed apps such as NGINX, Prometheus, WAF etc...). We should leverage the facts that Elastic collects all of the logs from all namespaces from our clusters in a structured way, and expose additional fields so the user can see all available logs. We'll need to investigate and design the way to expose logs from K8s cluster, and gitlab managed apps (e.g. NGINX ingress, HAproxy), this should also answer the need to expose WAF logs. In this issue we should come up with the right proposal on where and how we should expose those logs
Why cant we simply expose all logs from all namespace to every user?
Since Elastic is installed on a cluster which can have multiple projects, a single developer should not be exposed to the shared resources across the cluster and should see only the logs which are relevant to its project. This is why the environment (which is a namespace) is the first selector in the log explorer, so when filtering by an environment the user is seen only the logs which are relevant to the project
Proposal
We should use the log explorer with the filtered search, and surface additional information about the GitLab mng apps.
Proposal 1
Based on @akohlbecker comment
Hi all, I attempted this last month while overhauling some related parts of the logging interface. Unfortunately it can't work with the current way the logs UI is built. Indeed, everything in that UI is built on the assumption that you'll select an environment within your project, there is no concept of cluster or namespace.
You could potentially overload the environment dropdown to add gitlab-managed-apps, but it would result in a lot of monkey patching, and wouldn't be very clear to the user. There is also no way to support group or instance level clusters this way.
The conclusion we reached with @dhershkovitch was that we need a logs tab in the clusters admin page. This view wouldn't have an environment dropdown and instead work with namespaces as a top-level construct.
Pros -
- There is a clear separation between managed apps and deployed apps
Cons -
- Having 2 logs explorers could be confusing from the customer side (which one should we go?)
- It is also affecting our flow, when drilling down from an incident or a metric chart
- Maintenance might be high - we may end up building features which works only for one of the log explorer
Proposal 2
Use a single log explorer, however, when a maintainer is accessing it the log explorer will be in an advance mode exposing additional filters exclusively for managed apps logs (e.g. the maintainer will not see namespace filter instead of environment).
Pros-
Using a single log explorer eliminates the confusing and allowing a single user flow
Cons-
2 personas with different roles that are accessing the same UI will see different information