[Feature flag] Enable use of rate limits
What
Enable the :gitaly_rate_limit
feature flag, which enables Gitaly to put a limit on the rate of incoming requests.
Owners
- Team: Gitaly
- Most appropriate slack channel to reach out to:
#g_create_gitaly
- Best individual to reach out to: jcaigitlab
Expectations
What release does this feature occur in first?
What are we expecting to happen?
When the request rate for an RPC bursts the limit set, we should see "too many requests" errors in Gitaly.
What might happen if this goes wrong?
Requests fail with the above error messages even when the rate of requests has not reached the configured maximum.
dashboard:
What can we monitor to detect problems with this?
Roll Out Steps
-
Enable on staging -
Is the required code deployed on staging? (howto) -
Enable on staging (howto) -
Add featureflagstaging to this issue (howto) -
Test on staging (howto) -
Verify the feature flag was used by checking Prometheus metric [ gitaly_feature_flag_checks_total
]
-
-
Enable on production -
Is the required code deployed on production? (howto) -
Enable on production in #production
(howto) -
Add featureflagproduction to this issue -
Verify the feature flag was used by checking Prometheus metric gitaly_feature_flag_checks_total
-
-
Default-enable the feature flag (optional, only required if backwards-compatibility concerns exist) -
Wait for release containg default-disabled feature flag. -
Change the feature flag to default-enabled (howto) -
Wait for release containing default-enabled feature flag.
-
-
Remove feature flag
Please refer to the documentation of feature flags for further information.
Edited by John Cai