<!-- Before adding details, please check if there is a more focused issue template available to address your needs. --> ## General Information - Related issue for context (if applicable): https://gitlab.com/groups/gitlab-org/-/epics/19253#note_2750022218 - [Networking and Incident Management handbook page](https://handbook.gitlab.com/handbook/engineering/infrastructure-platforms/production-engineering/networking-and-incident-management) - Service this relates to (if applicable): ~"Service::DuoWorkflow" ## Details There is more detail at https://gitlab.com/groups/gitlab-org/-/epics/19253#note_2750022218 but it seems like when we use Cloudflare as part of our Duo Agent Platform stack (to proxy gRPC BIDI streams) we're dropping messages (they never seem to reach the server). We have a feature flag `duo_workflow_cloud_connector_url` now to toggle cloudflare on and off and things seem to be much more reliable with it off. Here is how it fits into the stack I'm referring to: with Cloudflare: ``` User -> (websocket to gitlab.com) -> Workhorse -> (gRPC via cloud.gitlab.com to Runway LB) -> Duo Workflow Service ``` Without Cloudflare: ``` User -> (websocket to gitlab.com) -> Workhorse -> (gRPC via Runway LB directly) -> Duo Workflow Service ``` The feature flag is toggling between those 2 options. It's worth noting that the first websocket connection going to "[gitlab.com](http://gitlab.com)" is always using cloudflare.