### Problem to solve After we ingest logs into Elasticsearch, we would like to provide our users with a real-time and historical log in a compact display that will be available within the Gitlab UI ### Intended users * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) ### Further details * Logs will be streamed to Gitlab UI directly from Elasticsearch. * Logs will be presented in a console-like view. * Users should be able to conduct a text search * Users should have the ability to conduct infinite scroll * Users should have the ability to filter by parameters such as Environment, Namespace, Pod name @ameliabauerly, @mrincon @akohlbecker right now we have multiple issues on very long threads, Based on https://gitlab.com/gitlab-org/gitlab/issues/37255, Here is a recap # What we have today If you navigate into pod logs you will be presented with a terminal view that display logs from the kubernetes api, the terminal view consists out of 2 filters `Environment` & `Pod name` If you enable the `:enable_cluster_application_elastic_stack` feature flag, and install Elastic Stack on your cluster, you get the same level of functionality that the kubernetes api provides: - viewing logs in a terminal-type UI - filtering by environment and pod name # What we need to do now to have a usable MVC when Elasticsearch is enabled - Add dynamic filters - filters are: `environments` `namespace`, `pod_name` - Display logs in a table-style UI - No need for a new UI component in the MVC - We need to take advantage of the full width available, table should stretch to fill the screen - Arbitrary search terms are handled by ES' full text search capabilities: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html (the search bar will provide the incremental value) # What would be nice in the near future - Update logs to be sorted most recent first - most use cases of looking at logs is "what just happened?": you want to see most recent logs first - Poll every 10sec for new logs - re-do a query for all logs "since the last one currently displayed", and update the view - Make sure filter state is saved to the URL, so the view I'm currently looking at can be shared with a colleague - Add a time range picker next to the search bar - also essential for sharing, if I open your link in an hour it should show the same view - Support saving views behind an opaque identifier (see https://gitlab.com/gitlab-org/gitlab/issues/37255#note_249995404) - Reconsider the need for navigation helpers (scroll to top/bottom and refresh) ## Mockups | Terminal view with filters and a search bar | Option for a "All pod" view | Empty state (if required) | |---------------------------------------------|-----------------------------|---------------------------| |  |  |  |