Elasticsearch results jumping to last page gives HTTP 500
Problem to solve
Accessing higher pages in search results causes Elasticsearch to fault out by returning this error:
"Result window is too large, from + size must be less than or equal to: [10000] but was [36920]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."
Even though this can be fixed by tweaking Elasticsearch configuration, setting index.max_result_window
to a higher value probably won't perform well so we should really be looking at implementing an efficient pagination strategy.
Further details
A customer is running into this problem when accessing higher pages in Elasticsearch search results.
Proposal
We should change our currently Elasticsearch results pagination to use the scroll API.
Links / references
- https://gitlab.zendesk.com/agent/tickets/109619 (internal only)
/cc @nick.thomas
Edited by John McGuire