Rate limit merge request Changes page diff resources
What does this MR do and why?
Adds a rate limit to the three diff resources the merge request Changes page loads:
GET /:namespace/:project/-/merge_requests/:iid/diffs_streamGET /:namespace/:project/-/merge_requests/:iid/diffs_statsGET /:namespace/:project/-/merge_requests/:iid/diff_files_metadata
For anonymous callers these run a live Gitaly compare instead of reading the stored diff, and they have no application rate limit today. This is the follow-up to !268216 (merged), which limits the Changes page itself, and it uses the same design:
- 360 requests per minute, counted per user for signed-in callers and per IP address for anonymous callers. The threshold evidence is in an internal note on this MR.
- One bucket shared by the three resources.
- Configurable with the new
merge_request_diff_resources_request_limitapplication setting (0turns it off). - Log mode first: a
staged_rulebehind themerge_request_diff_resources_rate_limitflag (gitlab_com_derisk, off by default). It's checked against the user when signed in, and the instance otherwise. While the flag is disabled, requests over the limit are logged but not blocked.
The design (per IP / per user, no shared per-project or per-MR buckets) matches !268216 (merged), !270001 and the REST diff routes in https://gitlab.com/gitlab-org/gitlab/-/work_items/606717.
Implementation and history
- The shared check is
Projects::MergeRequests::ApplicationController#check_diff_resources_rate_limit!. Thebefore_actions are inProjects::MergeRequestsController(diffs_stats,diff_files_metadata) andProjects::MergeRequests::DiffsStreamController(diffs_stream). - It runs after the inherited merge request lookup and read authorization.
- Specs cover:
- per IP across projects and MRs
- per user across IPs
- the shared bucket across the three resources
- log mode with the flag off
429with the flag on
- History: the first version of this MR used an anonymous-only, per-project key. It was reworked to per IP / per user, so the earlier review comments and the PM thread about the threshold refer to that version.
References
- Issue: https://gitlab.com/gitlab-org/gitlab/-/work_items/635093
- Changes page limit: !268216 (merged)
- Rollout issue: #639415
Screenshots or screen recordings
Not applicable: no UI changes.
How to set up and validate locally
-
In the Rails console, set a low limit:
ApplicationSetting.current.update!(merge_request_diff_resources_request_limit: 2) -
Without signing in, request the resources of a public merge request:
for path in diffs_stats diff_files_metadata diffs_stream; do curl -s -o /dev/null -w "%{http_code}\n" "http://gdk.test:3000/<namespace>/<project>/-/merge_requests/<iid>/$path" doneWith the flag disabled, all three return
200, and the third request's log line hasrate_limit_statecontaininglimit_mr_diff_resources_by_user_or_ip_log:log. -
Enable the flag with
Feature.enable(:merge_request_diff_resources_rate_limit)and repeat after a minute. The third request returns429, because the three resources share one bucket. A signed-in user gets their own bucket.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.