Skip to content
Snippets Groups Projects

Add enforce_ci_builds_pagination_limit feature flag

Merged Stan Hu requested to merge sh-ci-builds-pagination-limit-ff into master
1 unresolved thread

What does this MR do and why?

Previously enforcement of keyset pagination for Ci::Build requests was not active. This commit adds a feature flag that will allow enforcement of offset pagination limits to prevent slow database queries.

Relates to https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/24660

How to set up and validate locally

  1. Check out this branch.
  2. Log in to your GitLab instance.
  3. In the browser, try to access: http://gdk.test:3000/api/v4/projects/1/jobs?page=600&per_page=100. This should return empty results (assuming you don't have thousands of builds).
  4. In bin/rails console, run Feature.enable(:enforce_ci_builds_pagination_limit, Project.find(YOUR-PROJECT-ID).
  5. Wait 60 seconds or so and repeat step 3. This should result in an error:
{"error":"Offset pagination has a maximum allowed offset of 50000 for requests that return objects of type Ci::Build. Remaining records can be retrieved using keyset pagination."}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Stan Hu

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Contributor
    2 Warnings
    :warning:

    featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.

    For more information, see:

    :warning: This Merge Request needs to be labelled with feature flagexists, feature flag, backend. Please request a reviewer or maintainer to add them.

    Reviewer roulette

    Changes that require review have been detected!

    Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

    Category Reviewer Maintainer
    backend @Alexand profile link current availability (UTC+0, 7 hours ahead of author) @mcelicalderonG profile link current availability (UTC-5, 2 hours ahead of author)
    ~"Verify" Reviewer review is optional for ~"Verify" @iamricecake profile link current availability (UTC+8, 15 hours ahead of author)

    Please check reviewer's status!

    • available Reviewer is available!
    • unavailable Reviewer is unavailable!

    Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

    To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

    Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • Stan Hu marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

    marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

  • Stan Hu requested review from @j_lar and @mbobin

    requested review from @j_lar and @mbobin

  • Stan Hu added 1 commit

    added 1 commit

    • be86c327 - Add enforce_ci_builds_pagination_limit feature flag

    Compare with previous version

  • Stan Hu resolved all threads

    resolved all threads

  • Contributor

    Allure report

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :white_check_mark: test report for be86c327

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Data Stores | 20     | 0      | 0       | 0     | 20    | ✅     |
    | Govern      | 43     | 0      | 2       | 0     | 45    | ✅     |
    | Manage      | 0      | 0      | 1       | 0     | 1     | ➖     |
    | Plan        | 51     | 0      | 0       | 0     | 51    | ✅     |
    | Create      | 32     | 0      | 5       | 0     | 37    | ✅     |
    | Verify      | 20     | 0      | 0       | 0     | 20    | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 166    | 0      | 8       | 0     | 174   | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+

    e2e-package-and-test: :exclamation: test report for be86c327

    expand test summary
    +-------------------------------------------------------------+
    |                       suites summary                        |
    +--------+--------+--------+---------+-------+-------+--------+
    |        | passed | failed | skipped | flaky | total | result |
    +--------+--------+--------+---------+-------+-------+--------+
    | Verify | 100    | 0      | 10      | 2     | 110   | ❗     |
    +--------+--------+--------+---------+-------+-------+--------+
    | Total  | 100    | 0      | 10      | 2     | 110   | ❗     |
    +--------+--------+--------+---------+-------+-------+--------+
  • Marius Bobin approved this merge request

    approved this merge request

  • :wave: @mbobin, thanks for approving this merge request.

    This is the first time the merge request has been approved. To ensure full test coverage, a new pipeline will be started shortly.

    For more info, please refer to the following links:

  • Marius Bobin enabled an automatic merge when the pipeline for 743cdbae succeeds

    enabled an automatic merge when the pipeline for 743cdbae succeeds

  • merged

  • Marius Bobin mentioned in commit fb3b8cc3

    mentioned in commit fb3b8cc3

  • Marius Bobin mentioned in merge request !135235 (merged)

    mentioned in merge request !135235 (merged)

  • Marius Bobin mentioned in issue #429453

    mentioned in issue #429453

  • added workflowstaging label and removed workflowcanary label

  • Please register or sign in to reply
    Loading