Advanced Search should use point in time + search_after parameter

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Problem to solve

We've had this bug #33040 (closed) that shows the problems that are caused by not using the scroll API in Elasticsearch.

How pagination works today

page (default 1) and per_page (default 20, max 200) params are supported in the base SearchService.

Those params are passed into: Search::ProjectService, Search::SnippetService, Search::GroupService, or Search::GlobalService to execute the search. The search is executed by returning an instance of Gitlab::Elastic::SearchResults and calling the objects method.

Elasicsearch paging is handled in two ways and the results are always a Kaminari paginatable array:

1. (legacy, proxy method) page and per_page are handled by the the Elasticsearch client by calling .page and .per
2. (query builder method) page and size formats are sent in the query to Elasticsearch

Proposal

NOTE: This should be introduced behind a feature flag (with either a root ancestor group or user actor) to allow testing before release and a quick rollback if there is an issue.

IN PROGRESS

Advanced search can use Point in Time + search_after to allow searching past 10_000 records

https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html#point-in-time-api-example