Skip to content
Snippets Groups Projects

Add enforce_ci_builds_pagination_limit feature flag

Merged Stan Hu requested to merge sh-ci-builds-pagination-limit-ff into master

What does this MR do and why?

Previously enforcement of keyset pagination for Ci::Build requests was not active. This commit adds a feature flag that will allow enforcement of offset pagination limits to prevent slow database queries.

Relates to https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/24660

How to set up and validate locally

  1. Check out this branch.
  2. Log in to your GitLab instance.
  3. In the browser, try to access: http://gdk.test:3000/api/v4/projects/1/jobs?page=600&per_page=100. This should return empty results (assuming you don't have thousands of builds).
  4. In bin/rails console, run Feature.enable(:enforce_ci_builds_pagination_limit, Project.find(YOUR-PROJECT-ID).
  5. Wait 60 seconds or so and repeat step 3. This should result in an error:
{"error":"Offset pagination has a maximum allowed offset of 50000 for requests that return objects of type Ci::Build. Remaining records can be retrieved using keyset pagination."}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Stan Hu

Merge request reports

Merged results pipeline #1050074760 passed with warnings

Pipeline: TRIGGERED_EE_PIPELINE

#1050079218

    Pipeline: E2E GDK

    #1050082744

      Pipeline: Ruby 3 forced pipeline

      #1050078682

        +2

        Merged results pipeline passed with warnings for 743cdbae

        Test coverage 82.47% from 2 jobs

        Merged by Marius BobinMarius Bobin 1 year ago (Oct 26, 2023 8:46am UTC)

        Loading

        Pipeline #1050159689 passed

        Pipeline passed for fb3b8cc3 on master

        Test coverage 66.75% from 2 jobs
        10 environments impacted.

        Activity

        Filter activity
        • Approvals
        • Assignees & reviewers
        • Comments (from bots)
        • Comments (from users)
        • Commits & branches
        • Edits
        • Labels
        • Lock status
        • Mentions
        • Merge request status
        • Tracking
      • 2 Warnings
        :warning:

        featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.

        For more information, see:

        :warning: This Merge Request needs to be labelled with feature flagexists, feature flag, backend. Please request a reviewer or maintainer to add them.

        Reviewer roulette

        Changes that require review have been detected!

        Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

        Category Reviewer Maintainer
        backend @Alexand profile link current availability (UTC+0, 7 hours ahead of author) @mcelicalderonG profile link current availability (UTC-5, 2 hours ahead of author)
        ~"Verify" Reviewer review is optional for ~"Verify" @iamricecake profile link current availability (UTC+8, 15 hours ahead of author)

        Please check reviewer's status!

        • available Reviewer is available!
        • unavailable Reviewer is unavailable!

        Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

        To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

        Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

        If needed, you can retry the :repeat: danger-review job that generated this comment.

        Generated by :no_entry_sign: Danger

        Edited by Ghost User
      • Stan Hu marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

        marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

      • Stan Hu requested review from @j_lar and @mbobin

        requested review from @j_lar and @mbobin

      • Stan Hu added 1 commit

        added 1 commit

        • be86c327 - Add enforce_ci_builds_pagination_limit feature flag

        Compare with previous version

      • Stan Hu resolved all threads

        resolved all threads

      • Allure report

        allure-report-publisher generated test report!

        e2e-test-on-gdk: :white_check_mark: test report for be86c327

        expand test summary
        +------------------------------------------------------------------+
        |                          suites summary                          |
        +-------------+--------+--------+---------+-------+-------+--------+
        |             | passed | failed | skipped | flaky | total | result |
        +-------------+--------+--------+---------+-------+-------+--------+
        | Data Stores | 20     | 0      | 0       | 0     | 20    | ✅     |
        | Govern      | 43     | 0      | 2       | 0     | 45    | ✅     |
        | Manage      | 0      | 0      | 1       | 0     | 1     | ➖     |
        | Plan        | 51     | 0      | 0       | 0     | 51    | ✅     |
        | Create      | 32     | 0      | 5       | 0     | 37    | ✅     |
        | Verify      | 20     | 0      | 0       | 0     | 20    | ✅     |
        +-------------+--------+--------+---------+-------+-------+--------+
        | Total       | 166    | 0      | 8       | 0     | 174   | ✅     |
        +-------------+--------+--------+---------+-------+-------+--------+

        e2e-package-and-test: :exclamation: test report for be86c327

        expand test summary
        +-------------------------------------------------------------+
        |                       suites summary                        |
        +--------+--------+--------+---------+-------+-------+--------+
        |        | passed | failed | skipped | flaky | total | result |
        +--------+--------+--------+---------+-------+-------+--------+
        | Verify | 100    | 0      | 10      | 2     | 110   | ❗     |
        +--------+--------+--------+---------+-------+-------+--------+
        | Total  | 100    | 0      | 10      | 2     | 110   | ❗     |
        +--------+--------+--------+---------+-------+-------+--------+
        Edited by Ghost User
      • Marius Bobin approved this merge request

        approved this merge request

      • :wave: @mbobin, thanks for approving this merge request.

        This is the first time the merge request has been approved. To ensure full test coverage, a new pipeline will be started shortly.

        For more info, please refer to the following links:

      • Marius Bobin enabled an automatic merge when the pipeline for 743cdbae succeeds

        enabled an automatic merge when the pipeline for 743cdbae succeeds

      • Loading
      • Loading
      • Loading
      • Loading
      • Loading
      • Loading
      • Loading
      • Loading
      • Loading
      • Loading
      • Please register or sign in to reply
        Loading