Skip to content

Add feature flag for Duo Workflow streaming during tool calls

What does this MR do and why?

Add feature flag for Duo Workflow streaming during tool calls

This is related to gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!3459 (merged)

This introduces a feature flag to control new Duo Workflow Service behaviour introduced in that MR.

The problem we're trying to solve is that sometimes the Duo chat needs to spend a long time constructing a tool call (e.g. creating an MR) and while this is happening the connections (browser -> workhorse -> Duo Workflow Service) are all idle. This is leading to timeouts.

With this change we'll be able to keep streaming updates down as the tool call is being constructed so that the streams don't go idle.

We could have implemented a much narrower keepalive mechanism as described in #570575 (closed) but this is not so simple in practice. Keepalives require additional threads that are correctly synchronized with the actual workflow running threads and we need extra mutexes and error handling to make sure that we don't make concurrent use of gRPC or websocket connections.

This approach of, instead, streaming as the tool call is running potentially unblocks better user feedback opportunities in the future as shown in gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!3460 (closed)

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #570575 (closed)

Merge request reports

Loading