Loading
Gate wider packages API rate-limit path behind a flag
What does this MR do and why?
Adds a wider packages API path regex behind the gitlab_com_derisk flag packages_api_throttle_extended_paths, so the package list, detail, delete, pipelines and package_files routes count under the packages API rate limits instead of the general API limits. This is step 0 of the rollout in https://gitlab.com/gitlab-org/gitlab/-/work_items/634703#note_4151944429.
With the flag off, nothing changes: packages_api_path? checks the existing API_PATH_REGEX first.
Why
API_PATH_REGEXneeds a format segment afterpackages/, so/api/v4/projects/:id/packages(2.1M requests a day on GitLab.com),/packages/:package_id,/pipelinesand/package_filesare not package paths today. The Cloudflare bypass rule does cover them, so once that rule is removed they would fall to the general API limits (2,000 / 500) unless this flag is on./api/v4/groups/:id/packagesis deliberately not matched. It is under the general API throttle today, and moving it would loosen its limit.ThrottleRegistry.metaandLimiters.allare memoized, so a flag cannot change a staticpath:matcher. The packages rules now match a per-requestpackages_apifact, the same waydependency_proxyworks. Rack::Attack calls the samepackages_api_path?, so both stacks classify the same way.Feature.enabled?runs only for paths the old regex misses and the new one matches, so other requests skip the flag lookup.- One change in when the check runs: the old
path:matcher only ran when LabKit reached the packages rule, andbypass_headeris the first rule on every limiter, so bypassed requests never ran it. Thepackages_apifact is computed for every request whenlabkit_factsis built, asdependency_proxyalready is. That costs up to two regex matches per request, plus oneFeature.enabled?for the extended-only paths: about 2.16M requests a day on GitLab.com (about 25 per second), including while the edge bypass is in place. Flipper's per-request and in-process caches keep that cheap. - The regex ships to self-managed. The flag defaults off, so self-managed behaviour is unchanged until the flag is removed.
- On GitLab.com, once the edge bypass is removed, disabling this flag makes those routes stricter, not looser. The flag description and rollout issue say so.
References
- https://gitlab.com/gitlab-org/gitlab/-/work_items/634703
- Rollout issue: #638312
Screenshots or screen recordings
Not applicable: no UI change.
How to set up and validate locally
- In the Rails console, check the package list path with the flag off:
req = ->(path) { Gitlab::RackAttack::LabkitRateLimit::ClassifiedRequest.new(Rack::MockRequest.env_for(path)) } Feature.disable(:packages_api_throttle_extended_paths) req.call('/api/v4/projects/1/packages').labkit_facts[:packages_api] # => false req.call('/api/v4/projects/1/packages/npm/foo').labkit_facts[:packages_api] # => true - Enable the flag and check again:
Feature.enable(:packages_api_throttle_extended_paths) req.call('/api/v4/projects/1/packages').labkit_facts[:packages_api] # => true req.call('/api/v4/groups/1/packages').labkit_facts[:packages_api] # => false
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Sylvia Shen