Gate wider packages API rate-limit path behind a flag

What does this MR do and why?

Adds a wider packages API path regex behind the gitlab_com_derisk flag packages_api_throttle_extended_paths, so the package list, detail, delete, pipelines and package_files routes count under the packages API rate limits instead of the general API limits. This is step 0 of the rollout in https://gitlab.com/gitlab-org/gitlab/-/work_items/634703#note_4151944429.

With the flag off, nothing changes: packages_api_path? checks the existing API_PATH_REGEX first.

Why
  • API_PATH_REGEX needs a format segment after packages/, so /api/v4/projects/:id/packages (2.1M requests a day on GitLab.com), /packages/:package_id, /pipelines and /package_files are not package paths today. The Cloudflare bypass rule does cover them, so once that rule is removed they would fall to the general API limits (2,000 / 500) unless this flag is on.
  • /api/v4/groups/:id/packages is deliberately not matched. It is under the general API throttle today, and moving it would loosen its limit.
  • ThrottleRegistry.meta and Limiters.all are memoized, so a flag cannot change a static path: matcher. The packages rules now match a per-request packages_api fact, the same way dependency_proxy works. Rack::Attack calls the same packages_api_path?, so both stacks classify the same way.
  • Feature.enabled? runs only for paths the old regex misses and the new one matches, so other requests skip the flag lookup.
  • One change in when the check runs: the old path: matcher only ran when LabKit reached the packages rule, and bypass_header is the first rule on every limiter, so bypassed requests never ran it. The packages_api fact is computed for every request when labkit_facts is built, as dependency_proxy already is. That costs up to two regex matches per request, plus one Feature.enabled? for the extended-only paths: about 2.16M requests a day on GitLab.com (about 25 per second), including while the edge bypass is in place. Flipper's per-request and in-process caches keep that cheap.
  • The regex ships to self-managed. The flag defaults off, so self-managed behaviour is unchanged until the flag is removed.
  • On GitLab.com, once the edge bypass is removed, disabling this flag makes those routes stricter, not looser. The flag description and rollout issue say so.

References

Screenshots or screen recordings

Not applicable: no UI change.

How to set up and validate locally

  1. In the Rails console, check the package list path with the flag off:
    req = ->(path) { Gitlab::RackAttack::LabkitRateLimit::ClassifiedRequest.new(Rack::MockRequest.env_for(path)) }
    Feature.disable(:packages_api_throttle_extended_paths)
    req.call('/api/v4/projects/1/packages').labkit_facts[:packages_api]          # => false
    req.call('/api/v4/projects/1/packages/npm/foo').labkit_facts[:packages_api]  # => true
  2. Enable the flag and check again:
    Feature.enable(:packages_api_throttle_extended_paths)
    req.call('/api/v4/projects/1/packages').labkit_facts[:packages_api]          # => true
    req.call('/api/v4/groups/1/packages').labkit_facts[:packages_api]            # => false

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sylvia Shen

Merge request reports

Loading
Loading