Application logging (aggregated)
Part of Auto DevOps https://gitlab.com/gitlab-org/gitlab-ce/issues/35712
Auto log automatically gathers and searches the logs of applications that are deployed with GitLab.
Just like our Prometheus monitoring, logging can be deployed per application, this way we can easily scale it to millions of applications.
- Use the Elastic Search cluster already needed for GitLab Search? Does logging need different tuning than search? Is it a problem that CE doesn't do search via Elastic Search?
- Should we use FluentD since it is a cloud native standard?
- How do we allow people to search the logs from GitLab?
- How much should we help with tuning and maintaining the Elastic Search installation?
- Should we only gather application logs or many more?
Current WIP proposal:
Before introducing search, we need to allow for the installation of a system that aggregates the logs. That will be the focus of the work in this issue for the short-term. Search will be addressed as part of a follow-up issue: #31621
UI Updates required as part of this issue:
- Introduce an "all pods" option on the pod selection dropdown:
- If a user selects the "all pods" view, they will see the time stamp, the pod name and the log entry. If possible, these pieces of information will be color-coded to allow for quicker parsing of the aggregated logs. Since syntax highlighting is a user preference, ideally we would utilize the user's selected syntax highlighting theme and apply it to the logs, as well. For reference, the yellow shown below is