Multiple pages for keyset pagination
Proposal
We have need to read a large number of items (in some cases, more than 50,000 items) from several paginated APIs on a regular basis. We currently utilize offset pagination, as it is the only pagination style that allows us to retrieve several pages of results in parallel, and single-stepping through pages would be unacceptable performance-wise. However, we understand that GitLab is pushing keyset-style pagination, as it provides better database performance; but that style of pagination only allows retrieval of a single page at a time, which would not be an acceptable level of performance for our tools.
What I would like to suggest is to augment keyset-style pagination with an extra parameter that allows the caller to request additional page links. To refresh everyone's memory: in keyset-style pagination, an additional response header is provided with a link that allows loading the next page of results. If this proposal were implemented, this additional parameter would ask the API to include additional links in the response; so if the parameter were set to "5", a total of 5 response headers containing links would be included, allowing those additional pages to be retrieved in parallel.
I would expect there to be some limit on the total number of page links that could be requested, but for our purposes, a reasonable limit would probably be somewhere in the region of 10. (I'd really rather 100, as that seems to result in an acceptable level of performance for our application, but I accept that that's probably asking too much for keyset pagination.)