Change issues from index to upsert for Elasticsearch
What does this MR do and why?
Changes issues from using index
to using upsert
in order to allow us to conditionally add embeddings.
A requirement for this is to have routing be part of the document source as explained in !153394 (merged). We do an additional check that routing is part of the as_indexed_json
whenever upsert is used and send an error to sentry if that isn't the case.
The change is behind a feature flag [Feature flag] Rollout of `elaticsearch_issue_u... (#461625).
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Ensure the
AddRoutingToIssues
migration was finished. - Enable the feature flag:
Feature.enable(:elaticsearch_issue_upsert)
- Create an issue, see that a new document is created
- Update an issue, see that the document is updated
- Change an issue's routing, see that a new document is created
- Create a manual reference with a different routing, e.g.:
::Elastic::ProcessBookkeepingService.track!(Gitlab::Elastic::DocumentReference.new(Issue, 50, 50, "other_routing")); Elastic::ProcessBookkeepingService.new.execute
and see that there are now two documents with the same id and different routings. - [Optional] repeat with the feature flag disabled and see the same behaviour.
Related to #454249 (closed)
Edited by Madelein van Niekerk