Loading
Bound group releases finder count and page queries
What does this MR do and why?
Context
Releases::GroupReleasesFinder sorts every release in a group's hierarchy through one recursive keyset query. The REST group releases API counts that query for X-Total, and pages walk past every earlier row, so on large groups the count and deep pages can time out. The figures and the agreed approach are in https://gitlab.com/gitlab-org/gitlab/-/work_items/638258#note_4182445673.
This MR
- The capped count reads the hierarchy's releases directly instead of counting the recursive query.
- When a caller loads a page, each project contributes only its first
offset + limitreleases in the requested order, read from the(project_id, released_at, id)index, and the page's rows are loaded byid. The finder's relation builds this query when it loads, because callers setlimitandoffsetafter the finder returns. Any other change to the relation, or no limit, keeps the plain query. - We don't reuse
Gitlab::Pagination::Offset::PaginationWithIndexOnlyScan: it needs the page and page size up front, GraphQL offsets aren't page multiples, and it loads rows by every order column rather than byid. - It covers the REST API, the group releases page and GraphQL
Group.releases. Rows and order don't change, and the Changelog entry comes with the flag-removal MR.
This keeps the count and deep pages of a group's releases from timing out on large groups.
Feature Flag: group_releases_bounded_queries
References
- https://gitlab.com/gitlab-org/gitlab/-/work_items/638258+
- https://gitlab.com/gitlab-org/gitlab/-/work_items/639012+
Database queries
See MR comments.
How to set up and validate locally
- Run this with a personal access token (
read_apiscope) for a group with releases in more than one project:Note thecurl --include --header "PRIVATE-TOKEN: <your_access_token>" "http://gdk.test:3000/api/v4/groups/<id>/releases?per_page=5&page=2"X-Totalheader and the releases returned. - In a Rails console, run
Feature.enable(:group_releases_bounded_queries)and wait a minute. - Run the
curlagain.X-Totaland the releases are the same, andlog/development.logshowsSELECT COUNT(*) FROM (SELECT 1 AS one FROM "releases" WHERE "releases"."project_id" IN (...) LIMIT 10001), then a query startingWITH "group_releases_page" AS MATERIALIZEDwithCROSS JOIN LATERALandLIMIT 10.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Leaminn Ma