Reduce the sensitivity of the unauthenticated connections rate limit
Summary
We recently expired all tokens with no expiration dates: https://about.gitlab.com/blog/2023/10/25/access-token-lifetime-limits/
During this activity, many users experienced IP blocks due to retrying authentication too frequently.
As part of temporary mitigation, via gitlab-com/gl-infra/k8s-workloads/gitlab-com!3590 (merged) we temporarily changed the values to:
maxretry: 30 -> 500
findtime: 180 -> 60
bantime: 3600 -> 900
Follow up actions
We need to consider what impact this temporary change has had on the platform as a whole. We should then consider:
- Keeping the limit if our research shows that we are able to absorb the additional unauthenticated traffic
- OR adapt a more appropriate limit. Previous limit was too low and was causing various edge issues, so we need to find what a more appropriate limit should be.
This needs to be completed by 2024-05-24.
Related Incident(s)
Originating issue(s): production#18003 (closed)
Associated Services
Corrective Action Issue Checklist
-
Link the incident(s) this corrective action arose from -
Give context for what problem this corrective action is trying to prevent re-occurring -
Assign a severity label (this is the highest sev of related incidents, defaults to 'severity::4') -
Assign a priority (this will default to 'Reliability::P4' but should match the severity of the related incident) -
Assign a service label
Edited by Marin Jankovski