[Feature flag] Web hook paid customer rate limit exemptions
Summary
The web_hooks_no_rate_limit
flag is not intended to be rolled out, however, it allows us the ability to exempt a single customer from the limits while we work with them further to avoid triggering the limit #365605 (comment 1001610559).
It can also be used to switch off rate-limiting of paid customer webhooks for all customers.
Owners
- Team: ~"group::integrations"
- Most appropriate slack channel to reach out to:
#g_ecosystem_integrations
- Best individual to reach out to:
@.luke
- PM:
@g.hickman
Stakeholders
Expectations
What are we expecting to happen?
Enabling the flag for a paid customer's root namespace will make them exempt from webhook rate limits.
When is the feature viable?
What might happen if this goes wrong?
What can we monitor to detect problems with this?
Consider mentioning checks for 5xx errors or other anomalies like an increase in redirects (302 HTTP response status)
What can we check for monitoring production after rollouts?
Consider adding links to check for Sentry errors, Production logs for 5xx, 302s, etc.
Enable flag on non-production environments
We need to do this in order to toggle the flag on for a single customer on prod.
-
/chatops run feature set --staging web_hooks_no_rate_limit true
-
/chatops run feature set --dev web_hooks_no_rate_limit true
Disable webhook rate limits for specific customer
-
This feature can be disabled for a specific customer by running the following Chatops command:
/chatops run feature set --group=<customer-top-level-namespace web_hooks_no_rate_limit true
Disable webhook rate limits for all paid customers
-
This feature can be disabled by running the following Chatops command:
/chatops run feature set web_hooks_no_rate_limit true
Clean up
-
/chatops run feature delete web_hooks_no_rate_limit --staging
-
/chatops run feature delete web_hooks_no_rate_limit --dev
-
/chatops run feature delete web_hooks_no_rate_limit