2020-04-14: Elevated errors for API
Summary
More information will be added as we investigate the issue.
Timeline
All times UTC.
2020-04-14
- 19:00 - Error rates for pages and API
- 19:01 - Alert EOC
- 19:06 - Alert resolved
API puma workers were saturated
Pgbouncer connections:
This was due to Patroni hosts 1-10 and 12 were all restarted in the span of seconds in 19:00. Reason for this is still undetermined.
Resources
- If the Situation Zoom room was utilised, recording will be automatically uploaded to Incident room Google Drive folder (private)
Edited by Cindy Pallares 🦉