Investigate Grafana Loki for log storage
Loki is a new log-aggregation system from Grafana that should integrate pretty well with some of the other things I'm already using (Prometheus and Grafana): https://github.com/grafana/loki
Probably easiest to wait for Grafana 6.0 release for native support, and isn't extremely necessary yet, but could be a nice piece to add eventually that could help with some monitoring/investigation capabilities. It could also be overkill, but worth looking into.