Revert "Merge branch 'tb-route-driven-chat-init-tier2' into 'master'"
Purpose of revert
Incident: gitlab-com/gl-infra/production#22875 (closed) (INC-13926, "68.9% of chat workflows are stopped", Sev-3).
Reverted !251389 (merged) on a time-correlation hypothesis: the MR shipped 3 Sep and made the client close the websocket (WORKHORSE_WEBSOCKET_CLOSE_1000) on thread select / new chat, which matched the dominant stop reason.
Outcome: hypothesis disproven. The revert reached production 2026-09-08 05:57 UTC; the stop rate did not move (68.9% in the 24h before → 62.8% in the 3h after). Root cause was upstream: Anthropic rate limiting on claude-fable-5-1 (8M input tokens/min) during a Fable usage spike, compounded by duo-workflow-svc hitting its max_instances: 64 ceiling in us-east1, which produced repeated infrastructure-initiated cancellations. The rate returned to normal ~15:00 UTC on 8 Sep after the traffic block, with this change still reverted.
Reapplied in !254161 (merged) (19.4). Remaining FE follow-up, independent of this incident: scope adoptRunningStream() to a workflow id so opening a blank chat does not stop an in-flight turn — issue to follow.
Checklist
- Create an issue to reinstate the merge request and assign it to the author of the reverted merge request. → Reinstated directly via !254161 (merged) (author @rfrederico, approved by @ealcantara); no separate issue needed.
- If the revert is to resolve a broken 'master' incident … (n/a)
- If the revert involves a database migration … (n/a)
- Add the appropriate labels before the MR is created.
Related issues and merge requests
- Incident: gitlab-com/gl-infra/production#22875 (closed)
- Reverted: !251389 (merged)
- Reapply: !254161 (merged)
- Original issue: #619497 (closed)