Skip to content

Bump the size of elastic query for getting migrations

Madelein van Niekerk requested to merge fix-bump-elastic-migration-limit into master

What does this MR do and why?

For elastic migrations, we store a record of migrations in an index in elasticsearch to determine the state of all migrations.

Bug

When querying for migrations, there is a size parameter passed with the query to elasticsearch which determines the number of documents to search on. This number is currently set to 25 but we are on migration number 26.

What this means

For any new migrations, the state will not be read correctly which will cause the migration to be run every time the migration worker runs.

In staging, the CreateUserIndex migration keeps running every 30 minutes because it is migration number 26:

Screenshot_2022-10-31_at_11.28.58

Fix

Bump the size to 1000 (the size can't be set to unlimited). Another approach could be to make two calls every time: (1) find the count using the count API, (2) query with size=count but I don't think we want to make two calls for every migration check until this proves to be a problem again.

Edited by Madelein van Niekerk

Merge request reports