Add Files API throttling to application settings

Merged Vasilii Iakliushin requested to merge 335075_add_files_api_throttling_settings into master

What does this MR do?

Contributes to #335075

Roadmap

  1. Database migration MR <== This MR
  2. Throttling configuration form in Admin Area MR: !68560 (merged)
  3. Rack attack configuration MR: !68561 (merged)
  4. Documentation update: !68645 (merged)

Why

We want to have an option to rate limit Files API requests.

Proposal

Extend application_settings to support rate-limit configuration for Files API.

Set limit to unauthenticated Files API requests to max 125 req. in 15 seconds.

Set limit to authenticated Files API requests to max 500 req. in 15 seconds.

Disable Files API rate-limiting by default

New rate-limit settings

Unauthorized Files API requests Authorized Files API requests
~8 req/s ~33 req/s
500 req/min 2000 req/min

Request per minute limitations match default API rate-limits: https://docs.gitlab.com/ee/user/gitlab_com/index.html#gitlabcom-specific-rate-limits. But the new 15 seconds window protects the Files API from request bursts.

Current MR

  • Adds migration to extend application_settings table
  • Allows to update throttling settings

Database

Migrate

== 20210819120243 AddThrottleFilesApiColumns: migrating =======================
-- add_column(:application_settings, :throttle_unauthenticated_files_api_requests_per_period, :integer, {:default=>125, :null=>false})
   -> 0.0060s
-- add_column(:application_settings, :throttle_unauthenticated_files_api_period_in_seconds, :integer, {:default=>15, :null=>false})
   -> 0.0016s
-- add_column(:application_settings, :throttle_authenticated_files_api_requests_per_period, :integer, {:default=>500, :null=>false})
   -> 0.0021s
-- add_column(:application_settings, :throttle_authenticated_files_api_period_in_seconds, :integer, {:default=>15, :null=>false})
   -> 0.0015s
-- add_column(:application_settings, :throttle_unauthenticated_files_api_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0023s
-- add_column(:application_settings, :throttle_authenticated_files_api_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0014s
== 20210819120243 AddThrottleFilesApiColumns: migrated (0.0152s) ==============

Rollback

== 20210819120243 AddThrottleFilesApiColumns: reverting =======================
-- remove_column(:application_settings, :throttle_authenticated_files_api_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0053s
-- remove_column(:application_settings, :throttle_unauthenticated_files_api_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0012s
-- remove_column(:application_settings, :throttle_authenticated_files_api_period_in_seconds, :integer, {:default=>15, :null=>false})
   -> 0.0012s
-- remove_column(:application_settings, :throttle_authenticated_files_api_requests_per_period, :integer, {:default=>500, :null=>false})
   -> 0.0018s
-- remove_column(:application_settings, :throttle_unauthenticated_files_api_period_in_seconds, :integer, {:default=>15, :null=>false})
   -> 0.0012s
-- remove_column(:application_settings, :throttle_unauthenticated_files_api_requests_per_period, :integer, {:default=>125, :null=>false})
   -> 0.0013s
== 20210819120243 AddThrottleFilesApiColumns: reverted (0.0161s) ==============

How to setup and validate locally (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Mayra Cabrera