## Summary Implement the `clarification_question` tool for Duo Chat using the simpler POC approach validated in the spike #599778. This iteration delivers a functional clarification round-trip that allows agents to ask users clarifying questions before proceeding with execution. The tool uses the existing tool-use infrastructure in Duo Workflow Service and labels user responses as `clarification_answer` messages. This iteration is intended to be shipped behind a feature flag and is suitable for demo and early validation purposes. ## Background The spike (#599778) validated **Option B (tool-use approach)** as the recommended path. A proof of concept demonstrated the end-to-end flow across three repositories: - **Duo Workflow Service**: [ai-assist!5487](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/5487) — `clarification_question` tool prototype - **GitLab Rails**: [gitlab!235781](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235781) — backend support for clarification answer message sub-type - **Duo UI**: [duo-ui!537](https://gitlab.com/gitlab-org/duo-ui/-/merge_requests/537) — clarification question widget {width="900" height="451"} ## Scope ### Tasks | Task | Stack | Complexity | Description | Production MR | |------|-------|------------|-------------|---------------| | Implement clarification question widget in Duo UI | Frontend | Medium | Custom widget to display question text and selectable answer options. See UX notes below. | https://gitlab.com/gitlab-org/duo-ui/-/merge_requests/537+ | | Create a `duo_chat_clarification_question_tool` feature flag in the Rails Monolith | Backend | Low | Register a feature flag to control the enablement of the `clarification_question` tool in Duo Workflow Service | https://gitlab.com/gitlab-org/gitlab/-/merge_requests/236966+ | | Implement `clarification_question` tool in Duo Workflow Service | Backend | Medium | Gated by the `duo_chat_clarification_question` tool feature flag. The tool simply outputs a question + proposed options for now. | | | Transform chat log in clients to handle clarification answer messages | Frontend | Low | Hide raw user messages that answer clarification questions; display them inline with the question widget instead | | | Update foundational agent configurations | Backend | Low | Tell agents (e.g., planner agent, chat agent) that the `clarification_question` tool is available | | ### Approach details - The agent calls invokes the `clarification_question` tool with a `question` and `options` as input parameters. In the initial iteration, the tool just JSON dumps these input parameters as output. - The frontend renders an interactive widget with selectable options. - The user's selected option is sent back as a user message. - The client identifies that the user's prompt is intended as an answer for the `clarification_question` tool and feeds it back into the agentic loop. - Questions are asked **sequentially** (one at a time). The agent may invoke the tool multiple times during a flow. ### UX requirements (from @emilybauman) These UX guidelines were established during the spike discussion and should be followed for this iteration: 1. **Reduce clicks**: Follow the mockup and UX notes from [duo-ui!537 comment](https://gitlab.com/gitlab-org/duo-ui/-/merge_requests/537#note_3363839102) to minimize interaction friction. 2. **Answered state**: Once the user selects an option, the widget collapses to show only the question and the selected answer. 3. **Sequential questions only**: Questions are asked one at a time. All-at-once would turn the interaction into a form that fights the conversational flow. 4. **Chat history behavior**: - Answered questions display the question + selected answer in a collapsed state. - If the user ignores the question and types something else, the question becomes non-interactive but stays visible in the history. 5. **"Other" option**: Keep it simple for v1. Let the agent figure out intent from the user's free-text message. No need to separate "other option" from "other instruction" yet. ### Out of scope - Asking multiple questions at once (parallel questions UI). - Explicit skip/dismiss UI with Esc behavior (deferred to iteration 2). - Extending the flow registry framework with a new human interaction type (deferred to iteration 2). - Separate "other option" vs "other instruction" inputs (deferred to iteration 2). ## Known risks 1. There is a non-zero probability that a model won't use the tool to ask a clarification question even when prompted. 2. The POC relies on the tool prompt to stop flow execution once a question is posed. This is non-deterministic and might introduce undesired behavior in flows. 3. The client-side logic to identify that a user prompt is an answer to a clarification question adds complexity. ## Proof of concept MRs - Duo Workflow Service: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/5487 - GitLab Rails: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235781 - Duo UI: https://gitlab.com/gitlab-org/duo-ui/-/merge_requests/537 ## Related - Spike: #599778