Elasticsearch index is not stable
As shown in https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1745 and https://gitlab.com/gitlab-org/gitlab-ee/issues/2016 (among others), the schema for our elasticsearch index tends to differ between elasticsearch clusters. It also differs between tests and production.
This leads to bugs and would be considered unacceptable for our SQL database.
I propose we:
- Create a single
.json
file which is the canonical schema for our elasticsearch indexes - Use this directly when creating new indexes, e.g., in
rake gitlab:elastic:create_empty_index
orrecreate_index
or in test runs. - Disable dynamic mapping, dynamic creation of new fields, etc., so unknown fields or bad types become errors, rather than silently ignored
This does an end-run around a lot of elasticsearch-model
functionality, but our elasticsearch functionality is split between Go and Ruby anyway.
Once this is done, we should probably ask users to reindex, to ensure they're all using the known schema.
It will also be a good time to implement https://gitlab.com/gitlab-org/gitlab-ee/issues/2021
Relevant links
/cc @vsizov