Rate limit REST runner job lists filtered to user projects

What does this MR do and why?

Context

GET /runners/:id/jobs lists only jobs from the user's projects for every user except administrators in Admin Mode and auditors. The database reads the runner's jobs and discards those outside the user's projects, so a user who can see few of a busy runner's projects makes each call read far more jobs than it returns. In this step, we rate limit those calls. It builds on !268904 (merged), which rate limits calls filtered by a completed status. The figures behind it are in https://gitlab.com/gitlab-org/gitlab/-/work_items/638239#note_4175478465.

This MR

  • Rate limits calls filtered to the user's projects at 330 per minute per user, on a new limit with its own application setting, admin field and settings API attribute. Over the limit, the endpoint returns 429 Too Many Requests. Administrators in Admin Mode and auditors are not limited, because the endpoint lists all of a runner's jobs for them.
  • Logs calls over the limit instead of blocking them until the feature flag enforces it, so the logs show who the limit would block before it blocks anyone.

This caps how often one user can make GitLab read through a runner's job history for jobs they cannot see.

Feature Flag: ci_rate_limit_rest_runner_job_list_project_filtered

References

Screenshots or screen recordings

Before After
before_mr_a_admin_ci_cd after_branch_admin_ci_cd

How to set up and validate locally

  1. As a non-administrator, create a project runner in a project where you have the Maintainer role, and run a job on it.
  2. In a Rails console:
    ApplicationSetting.current.update!(runner_job_list_project_filtered_limit_per_user: 1)
  3. Wait a minute, so the web server picks up the new setting.
  4. Run this twice, with the non-administrator's personal access token and the runner's ID:
    curl --include --header "PRIVATE-TOKEN: <your_access_token>" "http://gdk.test:3000/api/v4/runners/<id>/jobs"
    Both return 200. In log/api_json.log, the second request's rate_limit_state includes applimiter_runner_job_list_project_filtered:limit_runner_job_list_project_filters_by_user_log:log.
  5. Run Feature.enable(:ci_rate_limit_rest_runner_job_list_project_filtered) and wait a minute.
  6. Run the curl twice more. The second returns 429 with the header RateLimit-Name: runner_job_list_project_filtered.
  7. Run it twice with an administrator's token, with the admin_mode scope if Admin Mode is enabled. Both return 200.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Leaminn Ma

Merge request reports

Loading
Loading