Keyset pagination for `Project` based on `id` column
What does this MR do?
This MR prepares support for keyset pagination for the Projects
API endpoint based on the id
column.
The idea is to:
- Add filters
id_before
,id_after
as a regular filter to the endpoint -> extracted into !19949 (merged) - We already support
order_by=id
andsort=asc|desc
- That's all we need for keyset pagination!
The code for the pagination now only needs to do this:
- Based on the order/sort information, figure out how to describe the
next
page - Add a link to the next page to the response header
I'm separating two concerns:
- Translate the HTTP request information and handling the response (e.g. setting headers)
- Working out the pagination information for the next page
Issue: #32186 (closed)
Database queries
This is a database query example for this request:
GET /api/v4/projects?pagination=keyset&id_after=22345098&order_by=id&per_page=20
SELECT
"projects".*
FROM
"projects"
WHERE
"projects"."visibility_level" = 10
AND (id > 22345098)
AND "projects"."pending_delete" = FALSE
ORDER BY
"projects"."id" DESC
LIMIT 20
Plan: https://explain.depesz.com/s/n7A
Outlook
Going forward, the idea is to add similar filtering support for other columns. We will basically have to match the order_by
columns and allow filtering with respective %_after, %_before
filters in order to support keyset pagination on those columns. If we wanted to keep supporting all order directions, we will have to add filters and corresponding indexes for all order-by directions available (#36157 (closed)).
Once this is proven to work and an accepted direction we want to go at, we'll add this to all available endpoints.
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team