Cloudflare: Follow-up: Allow per-route (/ai,/auth) RPM
To simplify adding cloud.gitlab.com/auth
endpoint we agreed that we are fine using the same default RPM for both /ai
and /auth
endpoints: https://ops.gitlab.net/gitlab-com/gl-infra/config-mgmt/-/merge_requests/8472#note_209565
When we have a chance, we may consider updating the module logic to allow per-route limits as suggested by Cameron:
Getting the rate limit into the module might require making the
rate_limit_user_rpm
variable a map of numbers that uses the routing key as a key and the number value being the rate limit. Then you could specify a rate limit per key.
variable "rate_limit_user_rpm" {
type = map(number)
description = "Rate limit per user per minute with routing key as a key"
default = {
ai = 100,
auth = 50
}
}
It could be referenced like as
${var.rate_limit_user_rpm["auth"]}
It was the original intent - https://ops.gitlab.net/gitlab-com/gl-infra/config-mgmt/-/merge_requests/8354/diffs#62514948f85e5f82b9547332cab8211858bf9168_13_19 - but we needed to refactor the module to support more than one route. Hence, the decision was to descope custom RPM for the initial iteration to unblock other teams working on Code Suggestions Low Latency effort.
Then we should set 50 RPM to /auth
to match !153831 (merged)