Migrate elasticsearch configuration from custom scripts to terraform
Background
Our elasticsearch configuration is currently managed by a set of custom scripts in the runbooks repo.
Problem
This configuration is messy and error-prone:
- Requires coordinating changes across different files, sometimes different languages (as some of the config is in jsonnet).
- This makes it difficult to introduce new indices.
- There is no drift detection.
Proposal
Migrate from custom scripts to terraform. There is a newly released official terraform provider for elastic.
This would allow us to specify:
- cluster configuration
- index templates
- ILM policies
- users
In a way that is is more accessible and ensures we detect config drift.