### Goal 1. Make Duo Chat **support more contexts** across GitLab and **make it easier to add more context in the future** 1. Access to GitLab search, read and list files in the local project. 2. Retrieve issues, MRs, pipeline logs of the current project even without specifying the specific issue, MR, retrieve file contents from other repositories. 3. Retrieve multiple resources to answer questions more holistically 2. Make Duo Chat **take actions** 1. Edit and create files 2. Changes will be undoable by the user, or require the user to accept/reject the proposed changes 3. **Improve correctness** of Duo Chat output 1. Less Contextual Awareness Gaps (i.e. less picking the wrong context) 2. Less Conversational Context Loss 4. **Do not introduce regression** 5. Initial scope is IDE only but in this sub-project we will also prepare the backend to support the Chat in the Web UI Not a goal: * While this will make Duo Chat use the Workflow architecture as its backend, the merging of the Duo Chat and Duo Workflow user interfaces is not in scope for this Sub-team project. ### Details This Epic will track the work to achieve the proposal outlined in this [discussion issue](https://gitlab.com/gitlab-org/gitlab/-/issues/524895), which proposes the introduction of Duo Workflow agentic behaviour and architecture to the Duo Chat interface. This effort is driven by an cross functional AI Subteam composed of team members from Duo Chat and Duo Workflow. For further details review this [doc](https://docs.google.com/document/d/1FD3wN9YauBR0-isCZPWEjNk_9ONxapwgtIWPs6nQ3bY/edit?usp=drivesdk) and see the [#subteam-duo-chat-workflow-service](https://gitlab.enterprise.slack.com/archives/C08JFG280BX) slack channel At the time of writing, the Duo Workflow is not GA. It is also important to note that there are a number of [requirements](https://gitlab.com/gitlab-org/gitlab/-/issues/524895#note_2397298890) that need to be addressed for the Duo Workflow architecture to be a replacement of the existing architecture behind Duo Chat. The proposal is to complete the following tasks: - Strenghten Duo Workflow conversational capabilities - Add streaming - Startup improvements - Support for slash commands - Add system prompt to stay on topic - Connect the Duo Chat interface to the Duo Workflow engine in VSCode as an additional "Agent Mode" - Bring this capability to other IDEs - Add necessary support to integrate into Web UI (post-multi-threaded release) \*The biggest risk is for a full replacement is that the Workflow ends up reducing or limmiting the performance/capabilities of Duo Chat. In that case we may need to reassess the approach and see if a hybrid architecture of Chat vs Workflow can be integrated into the overall Duo Chat experience. - Fully replace the current Duo Chat architecture with the Workflow one in IDE environment ### Scope of Duo Chat capabilities In order to consider replacing the current architecture of Duo Chat with Duo Workflow for classic conversations, the following scope of features and capabilities should be accounted for in the new architecture: <table> <tr> <th>Feature</th> <th>Scope</th> <th>Docs</th> <th>Workflow Support?</th> </tr> <tr> <td>Context Awareness</td> <td>Data entities in both the GitLab UI and IDEs (Issue, Epic, MR, Commit, Pipeline, Code)</td> <td> [Context Docs](https://docs.gitlab.com/user/gitlab_duo_chat/#the-context-chat-is-aware-of) </td> <td> [Context docs](https://docs.gitlab.com/user/duo_workflow/#the-context-workflow-is-aware-of) </td> </tr> <tr> <td>Ask Duo Chat</td> <td>Variety of questions that Chat can respond about data entities and code</td> <td> [Ask Duo Chat Examples](https://docs.gitlab.com/user/gitlab_duo_chat/examples/) </td> <td>On a first pass there shouldn't be any Workflow couldn't answer as well</td> </tr> <tr> <td>Multi-threaded</td> <td>Have conversations with Duo Chat in Multiple threads in GitLab UI and IDEs (and have a history of these conversations that is shared between all clients, so that conversations that start in one client can be continued in another client)</td> <td> [Multi-threaded conversations](https://docs.gitlab.com/user/gitlab_duo_chat/#have-multiple-conversations-with-chat) </td> <td>Duo Workflow is based on individual Workflow's by default, which can be thought of as individual threads</td> </tr> <tr> <td>Web IDE</td> <td>Use Duo Chat in Web IDE</td> <td> [Web IDE](https://docs.gitlab.com/user/gitlab_duo_chat/#use-gitlab-duo-chat-in-the-web-ide) </td> <td>Not yet supported.</td> </tr> <tr> <td>Extensions</td> <td>Use Duo Chat in IDE Extensions (VS Code, Visual Studio, JetBrains)</td> <td> [VS Code](https://docs.gitlab.com/user/gitlab_duo_chat/#use-gitlab-duo-chat-in-vs-code), [Visual Studio](https://docs.gitlab.com/user/gitlab_duo_chat/#use-gitlab-duo-chat-in-visual-studio-for-windows), [JetBrains](https://docs.gitlab.com/user/gitlab_duo_chat/#use-gitlab-duo-chat-in-jetbrains-ides) </td> <td>VSCode</td> </tr> <tr> <td>Slash Commands</td> <td> * Commands to control the chat: `/new`, `/help` * Commands to add context: `/incluude` * Commands to execute specific prompts: `/summarize_comments`, `/troubleshoot`, `/vulnerability_explain`, `/tests`, `/explain`, `/refactor`, `/fix` </td> <td> [Chat Slash Commands](https://docs.gitlab.com/user/gitlab_duo_chat/examples/#gitlab-duo-chat-slash-commands) </td> <td>Not yet supported.</td> </tr> <tr> <td>Access Control</td> <td>GitLab Duo Chat can be turned on and off, and availability changed, considering licensing, packaging and Unit Primitives</td> <td> [Chat Access Control](https://docs.gitlab.com/user/gitlab_duo_chat/turn_on_off/) [Duo Availability](https://docs.gitlab.com/user/gitlab_duo/turn_on_off/) </td> <td>Ultimate + experimental features enabled needed + Feature flag</td> </tr> <tr> <td>Metrics</td> <td>Measure detailed usage</td> <td> [AI Gateway Reporting](https://10az.online.tableau.com/t/gitlab/views/AIGatewayReporting/Overview/1ad1c3c1-5b8b-4b8f-bb15-f3049b7cf4e6/6966c9cc-ead1-4784-91ff-4705d7135a0d) </td> <td> [Duo Workflow Reporting](https://10az.online.tableau.com/#/site/gitlab/views/DuoRequestWorkflow/DuoWorkflow?%3Aiid=1&:isFromSaml=y) </td> </tr> <tr> <td>Model Support</td> <td>Duo Chat supports custom models, including those that may not have strong function calling capabilities.</td> <td> [Supported Models](https://docs.gitlab.com/administration/gitlab_duo_self_hosted/supported_models_and_hardware_requirements/#supported-models) </td> <td>Only Anthropic Claude Sonnet right now</td> </tr> <tr> <td>Deployment Model</td> <td>Duo Chat is available for SaaS (.com), Self Managed and Dedicated deployment models</td> <td></td> <td>Only SaaS</td> </tr> <tr> <td>Error Handling and Tracking</td> <td> Errors are explained to the user and ideal allow the user to self-help. Errors produced across the tech stack can be tracked depending on the stack layer where they are produced </td> <td> [Troubleshooting](https://docs.gitlab.com/user/gitlab_duo_chat/troubleshooting/) </td> <td> [Troubleshooting](https://handbook.gitlab.com/handbook/engineering/development/data-science/ai-powered/duo-workflow/troubleshooting/) </td> </tr> <tr> <td>Monitoring</td> <td>Monitoring (e.g. Grafana/Prometheus) and Logging (e.g. Kibana) is available for debugging and stack tracing purposes.</td> <td> [Runbook](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/duo-chat/README.md?ref_type=heads) [Error Budget Dashboard](https://dashboards.gitlab.net/d/stage-groups-detail-duo_chat/6c28d63a-60e8-5db3-9797-39f988a1900b?from=now-2d&orgId=1&to=now) </td> <td> [Service Monitoring](https://dashboards.gitlab.net/d/duo-workflow-main/duo-workflow3a-overview?from=now-6h&orgId=1&timezone=utc&to=now&var-PROMETHEUS_DS=mimir-runway&var-environment=gprd) </td> </tr> <tr> <td>Evaluation</td> <td>Duo Chat is evaluated regularly for quality of responses and to avoid regressions</td> <td> [Evaluation Runner](https://gitlab.com/gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/evaluation-runner#duo-chat) [ELI5](https://gitlab.com/gitlab-org/ai-powered/eli5) </td> <td> [Evaluation repo](https://gitlab.com/gitlab-org/duo-workflow/testing/duo-workflow-tests) </td> </tr> <tr> <td>NFR: Speed performance</td> <td>Speed of the new Chat should be better or equal to that of the legacy chat before switching</td> <td> [Current Goal](https://gitlab.com/groups/gitlab-org/-/epics/13866) </td> <td></td> </tr> <tr> <td>NFR: answer quality</td> <td>Quality of the answers of the new Chat need to be better or equal to that of the legacy chat before switching</td> <td> [Run a comparative user test](https://gitlab.com/gitlab-org/ux-research/-/issues/2513) </td> <td></td> </tr> <tr> <td>NFR: rate limit</td> <td>The rate limit is 160 calls per 8 hours per authenticated user.</td> <td> [Rate limits - AI Actions](https://docs.gitlab.com/security/rate_limits/#ai-action) </td> <td></td> </tr> <tr> <td>Roadmap: Global Search</td> <td>In the roadmap we will want to improve docs answers by leveraging Global Search</td> <td> [Discussion](https://docs.google.com/document/d/1KA4hD8t6PIhIyFlaZpYak5hGwsGTQt2zwAaopXFRho8/edit?usp=sharing) </td> <td></td> </tr> <tr> <td> Roadmap: https://gitlab.com/groups/gitlab-org/-/epics/16910+ </td> <td>In progress sub-team working to make Duo Chat aware of the codebase.</td> <td></td> <td></td> </tr> <tr> <td> Roadmap: https://gitlab.com/groups/gitlab-org/-/epics/16938+ </td> <td>Sub-team being established to enable users to customize the outputs of Duo features (including those of Chat).</td> <td></td> <td></td> </tr> </table> In the coming days we will continue to flesh out this Epic with some issues and a timeline once the Subteam has fully consolidated and kicked off the work. ### Deliverables and Timeline Starting from March 31st: - In 3 weeks: We will have ability for users to interact with an Agentic version of Duo Chat within the Duo Workflow interface in the IDE - In 5 weeks: We will have a new version of the Duo Chat UI living side by side with the old Duo Chat in the IDE sidebar. This new version will be the Agentic version but it does not require to access it via Workflow and it is connected to the new back-end architecture. (Some Duo Chat functions not available yet) - In 7 weeks: We will have feature parity with the old Duo Chat in the IDE. Except for support for self-managed, custom models. - \*No support for Custom Models - \*No Support for self-managed - \*Not replacement for Duo Chat in Web UI - In between these deliverables there will be some technical architecture milestones that will be reached: - Move the Workflow codebase into the same repo than AI Gateway. Both will be deployable as a single container. - Rewrite IDE executor in typescript, which will provide streaming capabilities and will provide faster load speed and resilience. * The items not yet supported at week 7 will become part of the individual group's roadmaps to work through to complete GA criteria (We will still update the timeline with that follow-up work in the coming weeks)  Related wiki page: https://gitlab.com/groups/gitlab-org/ai-powered/-/wikis/home/Agentic-IDE