Logging product direction
Update
Right now we have multiple issues on very long threads, I proposed to close all of them and coverage on a single meta issue, based on the related discussions so we can better track our progress.
Product direction
@dhershkovitch and I met today to discuss logging and what direction we want it to take in the near future.
There has been a lot of discussion lately, spread around in many issues, with varying levels of success, and keeping everyone on the same page has been challenging. I think having a synthesised snapshot of what we think we'll be doing in the next couple releases is going to be very helpful.
Please let me know your thoughts in the comments!
cc @mrincon @ameliabauerly @mnohr
What we have today
Today if you enable the :enable_cluster_application_elastic_stack feature flag, and install Elastic Stack on your cluster, you get the same level of functionality that the kubernetes api provides:
- viewing logs in a terminal-type UI
- filtering by environment and pod name
What we need to do now to have a usable MVC we can release
- Add a search bar with dynamic filters and arbitrary search terms
- filters are:
namespace,pod_name,container_name - arbitrary search terms are handled by ES' full text search capabilities: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html
- filters are:
- Update the top left dropdown to cluster rather than environment
- cluster is the base unit on which everything depends: do you have elastic stack installed or not
- selecting a cluster without ES installed should degrade smoothly into a view functionally equivalent to what we have today + display a prompt to install ES by linking to the documentation
- this is an important filter that should stay permanently displayed (ie: not part of dynamic filters)
- with the cluster selected you can get the values for dynamic filters in one K8S api call (namespaces, pods, containers)
- Display logs in a table-style UI
- for now we can do ASCII tables
2019-01-01T12:13:14:Z | pod-A | my log message - no need for a new UI component in the MVC
- we need to take advantage of the full width available, table should stretch to fill the screen
- for now we can do ASCII tables
Rough sketch of the interface
What would be nice in the near future
- Update logs to be sorted most recent first
- most use cases of looking at logs is "what just happened?": you want to see most recent logs first
- Poll every 10sec for new logs
- re-do a query for all logs "since the last one currently displayed", and update the view
- Make sure filter state is saved to the URL, so the view I'm currently looking at can be shared with a colleague
- Add a time range picker next to the search bar
- also essential for sharing, if I open your link in an hour it should show the same view
- Support saving views behind an opaque identifier (see #37255 (comment 249995404))
- Reconsider the need for navigation helpers (scroll to top/bottom and refresh)