Non-IP Rate Limiting Bypass Header logic is moved out of HAProxy
Problem
In https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/issues/25939 we moved the IP Based Rate Limiting Bypass Header logic out of HAProxy and into Cloudflare, to make management of customer allow lists easier.
There are four additional non-IP based bypasses in place, that we should also move before we mark this effort as complete, to minimise confusion, and ensure all bypass management is controlled in one place.
| Rule | Related Issue | Matcher |
|---|---|---|
|
|
https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/691+ |
Path (regex) |
|
|
https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/692+ -> Add separate RackAttack throttle for /jwt/auth (gitlab-org/gitlab#330107) |
Path |
|
|
https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/710+ |
Path (regex) |
|
|
https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/710+ |
Query string |
Related note.
Proposal
Create Cloudflare custom rules that set the X-GitLab-RateLimit-Bypass header for the following:
-
Path based: git -
Path based: jwt/auth -
Path: package registry -
Query string: go get
The logic should match the existing logic in HAProxy, and we should validate that the traffic patterns for any changes in traffic volumes. Example logs for /jwt/auth
-
Remove non-IP based bypass header logic from HAProxy