2025-10-11: rails_request error rate for ai-assisted main stage exceeding SLO

rails_request error rate for ai-assisted main stage exceeding SLO (Severity 4 (Low))

Problem: A spike in errors for the ai-assisted service was detected, with error rates peaking at 11.7% before returning closer to normal.

Impact: 170 errors from one user caused the ai-assisted service to exceed its error rate SLO, leading to temporary alerting and pages. No platform risk or broader customer impact identified.

Causes: A single new user made API requests that triggered authentication errors, likely due to insufficient OAuth token scopes for a specific endpoint.

Response strategy: We reviewed logs, identified the affected user and endpoint, merged related incidents, and resolved all alerts. A follow-up was created to investigate the errors with the owning team.

---

This ticket was created to track INC-4703, by incident.io 🔥