2025-10-10: rails_request error rate for ai-assisted main stage exceeding SLO
rails_request error rate for ai-assisted main stage exceeding SLO (Severity 3 (Medium))
Problem: Users were seeing HTTP 500 errors when trying to use the code search feature on GitLab.com. The ai-assisted service's error rate exceeded its SLO.
Impact: From 01:24 UTC, users on GitLab.com may have received HTTP 500 errors when using global code search or code suggestions. Disabling Zoekt search restored search functionality via Advanced Search fallback. After disabling a feature flag and re-enabling Zoekt, no new errors have been observed and all search features are now working as expected for users.
Causes: A recent merge request introduced a code change that led to HTTP 500 errors in the global code search feature. The problematic merge request has been identified and reverted.
Response strategy: We merged a revert for the problematic code change and picked it into auto-deploy branches. As a mitigation, we disabled Zoekt search, restoring search functionality by falling back to Advanced Search. After disabling a feature flag, we re-enabled Zoekt search in production. All search features are now operational and error rates have returned to normal.
This ticket was created to track INC-4676, by incident.io