New version of filebeat does not generate daily indices (dependency for WAF statistics)
After the most recent ELK version upgrade, indices changed from daily to a different heuristic as defined by ILM by default.
Test case:
- GKE setup on 2020.06.09
- On 2020.06.12 those are the available indices:
green open filebeat-7.6.2-2020.06.09-000001 Ku7WxTnwS1SprDBTiPfCYw 1 1 671424 0 1017.9mb 541.3mb
green open ilm-history-1-000001 z0vT1PgbRRSXaBX85UE2nQ 1 1 18 0 50.6kb 25.3kb
For any data range which doesn't not include 2020.06.09, the search (index based) will return empty.
for example:
While this one would accidentally return the 'right' results:
#input {"from":"2020-06-09T02:56:23.293Z","to":"2020-06-12T02:56:23.293Z"}
(from.to_date..to.to_date).map { |day| puts "filebeat-*-#{day.strftime('%Y.%m.%d')}-*"}
filebeat-*-2020.06.09-*
filebeat-*-2020.06.10-*
filebeat-*-2020.06.11-*
filebeat-*-2020.06.12-*
This one would return nothing (because 2020.06.09 contains all the data)
#input {"from":"2020-06-11T02:56:55.586Z","to":"2020-06-12T02:56:55.586Z"}
(from.to_date..to.to_date).map { |day| puts "filebeat-*-#{day.strftime('%Y.%m.%d')}-*"}
filebeat-*-2020.06.11-*
filebeat-*-2020.06.12-*