Bound group releases finder count and page queries

What does this MR do and why?

Context

Releases::GroupReleasesFinder sorts every release in a group's hierarchy through one recursive keyset query. The REST group releases API counts that query for X-Total, and pages walk past every earlier row, so on large groups the count and deep pages can time out. The figures and the agreed approach are in https://gitlab.com/gitlab-org/gitlab/-/work_items/638258#note_4182445673.

This MR

  • The capped count reads the hierarchy's releases directly instead of counting the recursive query.
  • When a caller loads a page, each project contributes only its first offset + limit releases in the requested order, read from the (project_id, released_at, id) index, and the page's rows are loaded by id. The finder's relation builds this query when it loads, because callers set limit and offset after the finder returns. Any other change to the relation, or no limit, keeps the plain query.
  • We don't reuse Gitlab::Pagination::Offset::PaginationWithIndexOnlyScan: it needs the page and page size up front, GraphQL offsets aren't page multiples, and it loads rows by every order column rather than by id.
  • It covers the REST API, the group releases page and GraphQL Group.releases. Rows and order don't change, and the Changelog entry comes with the flag-removal MR.

This keeps the count and deep pages of a group's releases from timing out on large groups.

Feature Flag: group_releases_bounded_queries

References

Database queries

See MR comments.

How to set up and validate locally

  1. Run this with a personal access token (read_api scope) for a group with releases in more than one project:
    curl --include --header "PRIVATE-TOKEN: <your_access_token>" "http://gdk.test:3000/api/v4/groups/<id>/releases?per_page=5&page=2"
    Note the X-Total header and the releases returned.
  2. In a Rails console, run Feature.enable(:group_releases_bounded_queries) and wait a minute.
  3. Run the curl again. X-Total and the releases are the same, and log/development.log shows SELECT COUNT(*) FROM (SELECT 1 AS one FROM "releases" WHERE "releases"."project_id" IN (...) LIMIT 10001), then a query starting WITH "group_releases_page" AS MATERIALIZED with CROSS JOIN LATERAL and LIMIT 10.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Leaminn Ma

Merge request reports

Loading
Loading