[[_TOC_]] ## Executive Summary With Flows creation, we are targeting non-technical Enterprise users in addition to technical users. Creating flows for Non-technical users is currently diffiult to impossible since it requires editing a YAML file with rigid syntax. To hit 1M requests in Q1—with 25% of Q1 goals coming from Custom Agents and Flows — the process to create a custom agents and flows must be intuitive and accessible. The proposed solution is to build a Visual UI flow editor (similar to n8n, OpenAI Agent Builder) that enables non-technical users to build flows without writing code. **Expected outcome:** An UI experience that makes it easy for non-technical Enterprise users to create custom agent flows that end users will repeatedly use on a continuous basis. --- ## Background ### Current State Currently, the only way to create a custom flow is through creating and editing a YAML file via the GitLab UI: {width="900" height="509"} The YAML configuration is stored at a path like `.gitlab/duo/flows/[flow-name].yaml` and defines the flow structure including agents, tools, and execution logic. ### Current Limitations This approach creates the following challenges: 1. **Accessibility barrier:** Non-technical users cannot create flows without learning YAML syntax 2. **Discoverability gap:** From the current UX, users cannot see which tools, components, agents, and sub-agents are available 3. **No visual feedback:** Users cannot visualize the flow logic, data paths, or agent interactions 4. **Error-prone:** YAML syntax errors are only caught at execution time, not during authoring 5. **Agent reuse limitation:** Users cannot currently invoke existing agents from the AI Catalog directly—they must be replicated in the YAML configuration ### Current GitLab Functionality to Surface in Visual Builder The visual builder must expose all existing GitLab Flows functionality. Based on current documentation, this includes: **Agent Types:** - **Foundational Agents** - Pre-built, production-ready agents created by GitLab (e.g., GitLab Duo Agent, Code Review, Software Development) - **Custom Agents** - User-created agents with custom system prompts and tool selections from the AI Catalog - **External Agents** - Agents integrated within DAP (Claude, Codex) as an option for users to pick from. **Current Triggers:** - **Mention** - When a service account user is @mentioned in a comment on an issue, merge request, or epic - **Assign** - When a service account user is assigned to an issue or merge request - **Assign reviewer** - When a service account user is assigned as a reviewer to a merge request **Configuration Sources:** - **AI Catalog** - Select from flows configured for the project - **Configuration path** - Reference a flow configuration file (e.g., `.gitlab/duo/flows/claude.yaml`) **Built-in Tools** - Create issue - Read file contents - Search code - Create merge request - And others as defined in the tool definitions **Execution Environment:** - Flows run in GitLab CI/CD pipelines - Configurable via `agent-config.yml` for custom Docker images, setup scripts, and caching - Uses composite identity combining user memberships and service account memberships **Sessions & Monitoring:** - Sessions track agent execution with status and logs - Viewable under **Automate \> Sessions** in the project - 30-day retention with activity reset - Display the auto-generated service account name when applicable (format: `ai-<flow>-<group>`) --- ## Main Goals 1. **Primary:** Create an experience that allows non-technical users to create flows within the scope of what is currently available in the platform (all current YAML capabilities must be representable visually) 2. **Secondary:** Create an extensible architecture that allows adding new functionality in the future based on customer input, including: - Additional trigger types (webhooks, scheduled, GitLab events) - Advanced routing and conditional logic - Multi-flow orchestration patterns --- ## Target Use Case ### Target Users 1. **Primary:** Non-technical Enterprise users (business analysts, operations teams, project managers) who want to automate workflows but cannot write YAML 2. **Secondary:** Technical users who want faster flow creation and easier visualization of complex workflows ### User Stories #### Canvas & Workspace - As a user, I want an infinite canvas where I can visually arrange and connect flow components - As a user, I want to easily discover which agents, tools, triggers, and other building blocks are available via a searchable node panel - As a user, I want to visualize the entire flow including agent inputs, outputs, logic branches, and data flow - As a user, I want zoom, pan, and minimap controls to navigate complex flows #### Flow Creation & Editing - As a user, I want to drag and drop components onto the canvas to build my flow - As a user, I want to connect nodes by drawing lines between outputs and inputs - As a user, I want to configure each node via a side panel with clear, labeled fields - As a user, I want to see the generated YAML code in real-time as I edit the visual builder (split view option) - As a user, I want inline validation that highlights errors before I try to run the flow - As a user, I want to be able to build flows with both concurrent and sequential nodes - As a user, I want to be able to pick within which groups and projects this flow is able to operate in. * NOTE: This should respect the RBAC rules and only show projects/groups that are available to the user #### Testing & Debugging - As a user, I want to run tests locally for quicker validation - Reference Ticket: [1714](https://gitlab.com/gitlab-org/editor-extensions/gitlab-lsp/-/work_items/1714%5D) - As a user, I want to see execution logs and step-by-step results in the editor - As a user, I want to execute individual nodes in isolation to debug specific steps `post-mvc` - As a user, I want to see which nodes succeeded/failed with clear visual indicators #### Saving & Governance - As a user, I want to save drafts of flows that are not yet active - As a user, I want to send flows for approval to group owners before enabling - As a user, I want version history so I can revert to previous configurations --- ## Functional Requirements ### 1. Triggers (Flow Entry Points) Triggers define how a flow is initiated. The visual builder must support: | Trigger Type | Description | Status | Reference Ticket | |--------------|-------------|--------|------------------| | **Mention** | Flow starts when service account is @mentioned in issue, MR, or epic comment | :white_check_mark: Exists | | | **Assign** | Flow starts when service account is assigned to an issue or MR | :white_check_mark: Exists | | | **Assign reviewer** | Flow starts when service account is assigned as MR reviewer | :white_check_mark: Exists | | | **Webhook/GL Events** | Flow starts when an HTTP endpoint receives a request | :arrows_counterclockwise: Pre-req | | | **Scheduled** | Flow starts on a cron schedule | :arrows_counterclockwise: Future | | | **End of another flow** | Flow starts when a parent flow completes | :arrows_counterclockwise: Future | | **UI Requirements:** - Each trigger type should have a distinct visual icon/shape - Configuration panel should show trigger-specific options (e.g., event filters for GitLab events) ### 2. Node Types #### 2.1 LLM Agent Nodes These represent AI agents that can reason and make decisions: | Node Type | Description | Configuration Options | |-----------|-------------|-----------------------| | **AgentComponent** | Generic LLM agent with custom prompt | System prompt, model selection, available tools | | **Foundational Agents** | Pre-built GitLab agents | Agent selection from AI Catalog, parameter overrides | | **Custom Agents** | User-created agents from AI Catalog | Agent selection, enable/disable in flow context | | **External Agents** | Third-party model providers | Provider selection, enable/disable in flow context | **Required UI Elements:** - System prompt text area (large, supports markdown preview) - Model selection dropdown (Claude Sonnet 4, etc.) - Tools multi-select from available built-in tools #### 2.2 Tool Nodes Deterministic steps that don't require LLM reasoning: _NOTE: Will need to come up with user friendly names for the "Node Types"_ | Node Type | Description | Reference Ticket | Labels | |-----------|-------------|------------------|--------| | **DeterministicStep** | Executes a specific action without AI decision-making | | | | **Built-in Tools** | GitLab native tools (Create Issue, Read File, Search Code, etc.) | | | | **Custom Tools** | User-defined tool implementations | | | | **MCP Server/Client** | External tools via Model Context Protocol Client | :arrows_counterclockwise: Pre-req | https://gitlab.com/groups/gitlab-org/-/work_items/20217 | **Required UI Elements:** - Tool browser/picker showing all available tools with descriptions - Per-tool configuration panel - Tool description field (critical for agent tool selection) #### 2.3 Logic & Control Flow Nodes | Node Type | Description | Visual Representation | Reference Ticket | |-----------|-------------|-----------------------|------------------| | **If/Else** | Conditional branch based on expression | Diamond shape with two outputs (true/false) | | | **Switch** | Multi-path router based on value | Diamond shape with multiple outputs | | | **Merge** | Combines multiple branches into single stream | Funnel shape | | | **Loop** | Iterates over array items | Circular arrow icon | | | **HITL (Human-in-the-Loop)** | Pauses for human approval before continuing | :arrows_counterclockwise: Pre-req | [1305](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/1305) | **Required UI Elements:** - Condition expression builder (visual and/or code mode) - Branch labeling - Clear visual distinction of paths (color-coding) ### 3. Connectors (Edges) Connectors show how data flows between nodes: **Requirements:** - Typed connections (prevent incompatible node connections) - Visual data flow direction (arrows, animation option) - Connection labels for complex flows - Support for multiple outputs from single node. - Visual feedback when dragging to create connections - Data preview on hover (show what data passed through) ### 4. Output & Response Nodes Define how the flow delivers results: | Output Type | Description | |-------------|-------------| | **GitLab Comment** | Posts response as comment on triggering issue/MR/epic | | **Merge Request** | Creates or updates an MR with changes | | **Webhook** | Sends data to external HTTP endpoint | | **GitLab Component** | Updates specific GitLab entity (issue description, labels, etc.) | **Output Format Options:** - Structured JSON with schema definition - Markdown (for rich text responses) - Plain text ### 5. Configuration Panel Side panel for configuring selected nodes: **Sections:** - **Parameters** - Node-specific settings (prompts, tools, conditions) - **Input Mapping** - How data from previous nodes maps to this node's inputs - **Output Mapping** - How this node's output is named/structured - **Advanced** - Timeout, retry logic, error handling behavior ### 6. Testing & Debugging Module **In-Editor Testing:** - "Run Flow" button to execute entire flow with test inputs - "Run from here" to execute from a specific node - Picker which data to use in test mode **Execution Visualization:** - Real-time node status indicators (pending, running, success, failed) - Step-by-step execution trace - Input/output data inspection at each node - Error messages with node highlighting ### 7. Version Control & Governance **Saving:** - Auto-save drafts - Manual save with optional commit message - Save as new flow (duplicate) **Versioning:** - Version history with diff view - Rollback to previous versions - Version pinning when enabled in groups/projects (align with AI Catalog versioning) **Approval Workflow:** - Draft → Pending Approval → Active states - Send for approval to group owners - Approval comments/feedback ### 8. YAML Code View **Requirements:** - Real-time YAML generation as canvas is edited - Split view option (canvas + YAML side-by-side) - YAML → canvas sync (edit YAML, update canvas) - Syntax highlighting and validation - Export/import YAML directly --- ## Pre-requisites (Dependencies on Other Teams) The following capabilities are being built by other teams and are required for full visual builder functionality: | Capability | Epic | Status | |------------|------|--------| | MCP Client Toolset | \[Link\] | :arrows_counterclockwise: Planned | | Human-in-the-Loop node | \[Link\] | :arrows_counterclockwise: In Progress | | Webhook Triggers | \[Link\] | :arrows_counterclockwise: Planned | | Scheduled Triggers | \[Link\] | :arrows_counterclockwise: Planned | | GitLab Event Triggers | \[Link\] | :arrows_counterclockwise: Planned | | MCP Enterprise Readiness | \[Link\] | :arrows_counterclockwise: Planned | | Governance Policies | \[Link\] | :arrows_counterclockwise: Planned | --- ## Success Metrics ### North Star **Credits consumed by custom flows** - The ultimate measure of value delivered through the visual builder and custom flows capability. ### Input Metrics Funnel | Stage | Metric | What it measures | |-------|--------|------------------| | **Creation** | Number of flows created | Top of funnel - are users building flows? | | **Activation** | Number of flows enabled (in groups/projects) | Created → actually deployed for use | | **Activation** | Time to first flow creation | Friction in onboarding experience | | **Usage** | Number of flows invoked | Direct driver of credit consumption | | **Usage** | Invocations per flow (distribution) | Are a few flows driving all usage, or broad adoption? | | **Usage** | Flows invoked per project | Penetration across customer base | | **Retention** | Repeat invocations per user | Stickiness - are users coming back? | | **Quality** | Flow success rate (completed vs failed sessions) | Failed flows = wasted credits, poor experience | | **Quality** | Average session duration | Proxy for flow complexity/value delivered | ### Funnel Relationship ``` Flows Created ↓ Flows Enabled ──→ Time to first creation (measures friction) ↓ Flows Invoked ──→ Invocations per flow (distribution) │ Flows invoked per project (breadth) ↓ Repeat Invocations ──→ Per user (retention signal) ↓ Credits Consumed (North Star) ↑ Flow success rate & session duration (quality multipliers) ``` ### Workflow Displacement Metrics These metrics measure the real-world impact of flows by tracking how much work shifts from human-led to flow-led processes. These metrics can potentially be made user/customer facing to help illustrate them the ROI using DAP Platform. cc @frwang1 : | Metric | Description | |--------|-------------| | **Displacement rate** | % of events now handled by flows vs. total events of that type | | **Human intervention rate** | % of flow executions requiring human follow-up (vs. fully autonomous resolution) | _Note: These metrics will vary by use case (pipeline remediation, code review, issue triage, security fixes, etc.) and should be tracked per flow category._ # Other requirements **Notification Requirements:** - Flow author notification settings per output type. Critical for background flows where user isn't watching --- ## Appendix Current Designs: [Figma Link](https://www.figma.com/design/8dIIoI6h78WTjXbUTp0yOI/Agent-Workflow-Catalog?node-id=6647-46080&t=OpLiyFM7CoYONfPQ-1)