Treat API requests from the frontend as web traffic in the rate limiter

What does this MR do and why?

This will allow us to impose stricter rate limits for general API traffic, without affecting interactive API requests made by the frontend during normal GitLab usage.

The frontend requests are identified by the inclusion of a CSRF token in the headers.

Other rate limits that only affect a subset of API requests (e.g. the Files and Packages APIs, or protected paths) still take precedence, i.e. requests for these paths will always be matched even if they include a CSRF token.

Issue: #344807 (closed), follow-up to #335300 (closed)

How to set up and validate locally

1. Enable rate limiting with a low threshold:

   update application_settings set throttle_unauthenticated_enabled = true, throttle_unauthenticated_api_enabled = true, throttle_unauthenticated_requests_per_period = 10, throttle_unauthenticated_api_requests_per_period = 10;

2. Apply settings with gdk restart rails-web (you can also reset the rate limit state at any time with gdk restart redis)
3. Without logging in, visit a page which makes API requests, e.g. http://localhost:3000/Commit451/lab-coat/-/issues/1
   • Resend one of the requests until you get a 429 response with a Ratelimit-Name: throttle_unauthenticated_web header:
4. Send API requests without a CSRF token, using e.g. curl -v http://localhost:3000/api/v4/groups
   • Repeat until you get a 429 response with a Ratelimit-Name: throttle_unauthenticated_api header
5. Reset rate limiting settings:

   update application_settings set throttle_unauthenticated_enabled = false, throttle_unauthenticated_api_enabled = false, throttle_unauthenticated_requests_per_period = 3600, throttle_unauthenticated_api_requests_per_period = 3600;

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.