Phase 2: Add automation to keep default limits in sync with GitLab.com settings
The purpose of this issue is to automate checks between the default limits set at the application level and the current GitLab.com settings. More context and details can be found in the parent Epic: &529
This phase can be completed after Phase 1 completes.
Steps
- Add a step/documentation to the on-call runbook so that whenever an SRE updates a rate limit setting on GitLab.com they also update the default instance setting.
- Build a job that pulls the current rate limit settings on .com using the api, compares them to the default application values, and alerts if there are any discrepancies. Can run the job periodically or triggered whenever a default value is altered. The limit settings we'd like to check and alert on for discrepancies are documented here: https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/13760#settings-to-update
- Consider making the output of the job public
Edited by Andrew Thomas