Add keyset pagination support for Projects API for all order-by/sort combinations

#32186 (closed) adds support to use keyset pagination on the Projects API on the primary key - which only works when also ordering by id (order_by=id). This issue is for reviewing other order_by options and making sure that

  • keyset pagination works for these columns, in particular with data types other than integer
  • pagination is performant and we have the right indexes in place to support this.

An alternative outcome of this issue is to determine which order_by options we won't be able to support efficiently with keyset pagination.

Proposal

  • Disallow offset based pagination above high offset (configurable, default 10,000)
  • Only provide keyset pagination based on id and no other order-by criteria. The main use case for this is to retrieve the full list of a very large collection, e.g. all projects.

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖