Add configurable search rate limits

Merged John Mason requested to merge generalize-user-email-lookup-rate-limit into master

What does this MR do and why?

This mirrors GitHub's API search rate limits and adds the following configurable rate limits to search endpoints:

  • Default 30 requests / minute for authenticated users
  • Default 10 requests / minute unauthenticated IP addresses.

There was an existing rate limit called user_email_lookup_limit that is now migrated to the authenticated search_rate_limit. This removes user_email_lookup_limit.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/344907

After this is merged, we can go and remove the current rate limits in Cloudflare.

Screenshots or screen recordings

Screenshot taken locally from http://localhost:3000/admin/application_settings/network#js-search-limits-settings

image

Screenshot of updated documentation:

image

How to set up and validate locally

Testing authenticated per-user rate limit in UI

  1. Run db migrations
  2. Sign in to local gdk gitlab (if you are not already signed in)
  3. Go to http://localhost:3000/admin/application_settings/network#js-search-limits-settings and set authenticated rate limit to something really low, like 3.
  4. Try searching more than the limit in the UI, such as this: http://localhost:3000/search?scope=projects&search=flight

Testing the unauthenticated per-ip rate limit in UI

  1. Run db migrations (if they have not already be run)
  2. Sign out of local gitlab instance
  3. Try searching more than the limit in the UI, such as this: http://localhost:3000/search?scope=projects&search=flight

Testing authenticated per-user rate limit in API

  1. Run db migrations (if they have not already be run)
  2. Sign in to local gdk gitlab (if you are not already signed in)
  3. Try searching more than the limit in the API, such as this: http://localhost:3000/api/v4/search?scope=projects&search=flight

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by John Mason