Support ascending keyset pagination for runner jobs

What does this MR do and why?

This MR adds ascending keyset pagination to GET /runners/:id/jobs.

The endpoint already supports cursor-based keyset pagination, but Ci::Build previously limited keyset ordering to descending IDs. Offset pagination can time out when listing runner jobs because later pages scan and discard preceding rows. It also issues a count query for the total headers.

Production testing used runner 32881696 on a thin database clone:

Query Execution time I/O read time Buffers
Existing ascending offset, page 11 327.737 ms 325.221 ms 8 hits, 317 reads
Ascending keyset, first page 0.368 ms 0 ms 107 hits
Existing descending keyset, first page, warm rerun 0.759 ms 0 ms 114 hits
Ascending keyset, page 11 equivalent 2.508 ms 1.653 ms 104 hits, 5 reads

Cache state differed between runs, so these results do not represent an exact speedup ratio.

The API documentation now recommends keyset pagination, shows how to follow the next-page link, and explains which total headers keyset responses omit.

References

Database query plans

The offset baseline used this representative query shape

https://console.postgres.ai/gitlab/projects/gitlab-production-ci/sessions/55398/commands/159060

SELECT p_ci_builds.*
FROM p_ci_builds
WHERE p_ci_builds.type = 'Ci::Build'
  AND p_ci_builds.runner_id = 32881696
  AND p_ci_builds.status = 'success'
ORDER BY p_ci_builds.id ASC
LIMIT 20 OFFSET 200;
The ascending keyset first page used this representative query shape

https://console.postgres.ai/gitlab/projects/gitlab-production-ci/sessions/55398/commands/159063

SELECT p_ci_builds.*
FROM p_ci_builds
WHERE p_ci_builds.type = 'Ci::Build'
  AND p_ci_builds.runner_id = 32881696
  AND p_ci_builds.status = 'success'
ORDER BY p_ci_builds.id ASC, p_ci_builds.partition_id ASC
LIMIT 21;
Ascending keyset, page 11 equivalent

https://console.postgres.ai/gitlab/projects/gitlab-production-ci/sessions/55398/commands/159062

SELECT p_ci_builds.*
FROM p_ci_builds
WHERE p_ci_builds.type = 'Ci::Build'
  AND p_ci_builds.runner_id = 32881696
  AND p_ci_builds.status = 'success'
  AND (p_ci_builds.id, p_ci_builds.partition_id) > (6212051867, 101)
ORDER BY p_ci_builds.id ASC, p_ci_builds.partition_id ASC
LIMIT 21;

Both keyset directions use the existing (runner_id, id DESC) btree in opposite scan directions. The plans use Merge Append across partitions and a small incremental sort for the partition_id tie-breaker. No new index is needed.

The Database Lab session contains the full EXPLAIN output and production evidence.

Screenshots or screen recordings

Not applicable - backend and API documentation change only.

How to set up and validate locally

  1. Run the model spec:

    bundle exec rspec spec/models/ci/build_spec.rb:2313 --format documentation
    # 1 example, 0 failures
  2. Run the request specs:

    bundle exec rspec spec/requests/api/ci/runners_spec.rb:2082 spec/requests/api/ci/runners_spec.rb:2093 --format documentation
    # 6 examples, 0 failures
  3. Run RuboCop:

    bundle exec rubocop -A spec/requests/api/ci/runners_spec.rb spec/models/ci/build_spec.rb app/models/ci/build.rb
    # Pass
  4. Lint the API documentation:

    scripts/lint-doc.sh doc/api/runners.md
    # Markdownlint and Vale pass

Commit hooks also pass, including the documentation link check.

MR acceptance checklist

  • Tests cover ascending and descending keyset orderings.
  • API documentation describes keyset pagination and response headers.
  • Database review is complete because the query touches the partitioned p_ci_builds table.
  • No database migration or new index is needed.
  • No feature flag is needed.
Edited by Pedro Pombeiro

Merge request reports

Loading
Loading