Duo Workflow - Experiment Releases
### Background
Following the successful completion of the [POC phase](https://gitlab.com/gitlab-org/ai-powered/ai-framework/team-hq/-/issues/45 "POC phase"), this epic encompasses the work streams and items required to develop and ship the first version of Duo Workflows. This epic will guide the project through its Dogfooding/Alpha, Closed Experimental, and Open Experiment phases.
### Key Components
1. Workflow Service: A Python-based engine orchestrating workflow logic and state management.
2. Workflow Executor: A runtime entity for safe action execution, either locally or in a CI environment.
3. Human-in-the-Loop Features: Enabling user interactions within workflows for enhanced control and oversight.
### Goals
1. [Dogfooding/Alpha (Internal)](https://gitlab.com/groups/gitlab-org/-/epics/15201 "Duo Workflow - Dogfooding/Alpha Release")
- Develop a version of Duo Workflow for internal testing and refinement.
- Generate interest and buy-in for GitLab's AI vision among team members.
2. [Closed Experiment](https://gitlab.com/groups/gitlab-org/-/epics/15046 "Duo Workflow - Closed Experiment Release")
- Create a version of Duo Workflows for limited customer testing.
- Gather initial feedback from select customers in controlled environments.
3. [Open Experiment](https://gitlab.com/groups/gitlab-org/-/epics/15200 "Duo Workflow - Open Experiment Release")
- Launch a publicly accessible version of Duo Workflows, labeled as an "Experiment".
- Collect broader user feedback and validate the solution's effectiveness.
### Focus Use Cases
Our primary focus for testing and user articulation will be:
1. Coding tasks
- Enable Duo Workflow to read issues / MRs and write code based on user goal- refactor, bootstrap a project, add functionality or tests
2. Enhancing code quality
- Improve test coverage, manage dependency upgrades, assist with refactoring, and aid in bug fixing.
Note: While optimizing for these use cases, Duo Workflow will remain flexible for other applications.
### Definition of Done
- Goal: Ship an experiment enabling observation of user reactions and engagement in a production setup.
- Primary Use Case: Coding tasks
- Focus: Introduce a first use case as an experiment, maintaining handcrafted agent usage.
- Rationale: Learning from actual usage is critical to understand user interactions, trust levels, and important features.
- Criteria: Successfully deploy a workflow that fixes a broken pipeline, operabl from the vscode IDE extensions, with sandboxed local execution, user confirmation for commands, documentation, and performance tracking.
### Release Iterations
1. Dogfooding/Alpha (17.4):
- Ship core Workflow Service and Executor
- Implement basic UI and API integrations
- Enable Docker support and GitLab checkpointing
2. Closed Experiment (17.5):
- Enhance Dev Container support
- Improve GitLab checkpointing UI and functionality
- Implement initial human-in-the-loop features
3. Open Experiment (17.6):
- Expand IDE integration and support
- Integrate with CI Pipeline and GitLab Workspaces
- Refine and optimize based on early user feedback
Upon completion of these iterations, we will officially enter the Experiment state and begin preparations for the Beta phase.
### Key Metrics and Success Criteria
- User engagement rates
- SWE-Bench
- User satisfaction scores
- Performance and reliability metrics
### Links
- [Duo Workflow Board](https://gitlab.com/groups/gitlab-org/-/boards/7545476?label_name%5B%5D=Category%3ADuo%20Workflow)
- [GitLab Experiment Support Policy](https://docs.gitlab.com/ee/policy/experiment-beta-support.html)
epic