Add API endpoint rate limit for virtual registries

David Fernandezrequested to merge
521692-vregs-rack-attack-limits into master

🗃️ Context

The maven virtual registry is a pull through cache where the GitLab instance is between a maven client (that pulls packages) and an upstream registry.

Thus, the maven virtual registry receives requests for files. Those requests are fulfilled with the help of the configured upstream.

In Maven Virtual Registry: Performance review (#468116 - closed), we noticed that some maven clients (namely sbt) can set up a situation where an 🌊 of file requests is done.

The problem is that the maven virtual registry API falls under the default rack attack authenticated (anynomous access is not allowed) limit. That limit is too low to what maven clients can do, thus, they can hit the 429 Too Many Requests response very easily.

Since this situation is very similar to what we have in the package registry and its dedicated limit, this MR aims to implement a dedicated rate limit for virtual registry endpoints (that are used by package manager clients). The other endpoints, like the Rest API to manage the virtual registries objects, are still under the authenticated API limit.

Since the package registry limit is working quite well, we're going to use the same one here. The default value of that limit is 1000requests / 15sec.

That new limit will be exposed in the UI and api so that self-managed admins can adjust the limit.

Lastly, the maven virtual registry feature is:

For the limit implementation, we are using this guideline that will be powered by a new attribute in the rate_limit jsonb column.

The related issue is #521692 (closed) .

🤔 What does this MR do and why?

  • Add a virtual_registries_endpoints_api_limit attribute to the rate_limits column in the application settings table.
  • Add a new EE ApplicationRateLimiter based on that attribute.
  • Update the application settings API/UI to expose the virtual_registries_endpoints_api_limit field.
  • Update the maven virtual registry endpoint (used by package managers) to enforce the application limit.
    • Disable the rack attack limits for those endpoints.
  • Update all the related specs.

📚 References

🖥️ Screenshots or screen recordings

CI/CD admin settings (/admin/application_settings/ci_cd):

Before After
Screenshot_2025-03-21_at_08.54.48 Screenshot_2025-03-31_at_09.23.27

⚙️ How to set up and validate locally

Setup:

  • Enable the feature flag in a rails console : Feature.enable(:virtual_registry_maven).
  • Make sure to have a GitLab license active as the maven virtual registry is an EE only feature.
  • Have a top level group ready. Any visiblity
  • Have a PAT ready. Scope api or read_virtual_registry.

Update the rate limit using the UI:

  • Browse Admin -> Settings -> CI/CD -> Virtual Registries.
  • Update the rate limit to 5 and save the settings.

Update the rate limit using the API:

  • You will need the PAT of an instance admin here.
  • Read the settings: 
    $ curl -H "Private-Token: <instance admin PAT>" "http://gdk.test:8000/api/v4/application/settings" | jq | grep "virtual"
"virtual_registries_endpoints_api_limit": 1000,
  • Update them: 
    $ curl -X PUT -H "Private-Token: <instance admin PAT>" "http://gdk.test:8000/api/v4/application/settings?virtual_registries_endpoints_api_limit=5" | jq | grep "virtual"
"virtual_registries_endpoints_api_limit": 5,

Next, let's set up a virtual registry using curl, the PAT and the top level group:

$ curl -X POST --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/groups/<group_id>/-/virtual_registries/packages/maven/registries" | jq
# note the registry id field

$ curl -X POST --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/virtual_registries/packages/maven/registries/<registry id>/upstreams?url=https://repo1.maven.org/maven2" | jq

Ok, everything is ready. Now, let's simulate 10 requests to the virtual registry endpoint (within 15 seconds).

$ repeat 10 curl --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/virtual_registries/packages/maven/32/org/springframework/spring-context/6.1.15/spring-context-6.1.15.pom" 2>/dev/null | grep -e "project xmlns" -e "Try again later"
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
{"message":{"error":"This endpoint has been requested too many times. Try again later."}}
{"message":{"error":"This endpoint has been requested too many times. Try again later."}}
{"message":{"error":"This endpoint has been requested too many times. Try again later."}}
{"message":{"error":"This endpoint has been requested too many times. Try again later."}}
{"message":{"error":"This endpoint has been requested too many times. Try again later."}}

Our rate limit is set to 5, so the first 5 requests received the xml document (<project xmlns ...) and the 5 other requests, they were greeted with Too many requests.

You can try with different rate limits or try to executed the above command very quickly, thus within the 15 seconds considered = only the first 5 requests are accepted and all the others will 💥.

The rate limit is thus behaving as expected 🎉

🏎️ MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

✅

Edited by David Fernandez

Merge request reports