## Summary A saved (and therefore released) custom flow that lists AI Catalog agents in its YAML's `include` entries will run them as subagents during execution. This epic covers the execution pathway. The save pathway was [gitlab-org&23526](https://gitlab.com/groups/gitlab-org/-/work_items/23526), now closed. Two items from it are children here. ### References - Parent epic: [gitlab-org&21832](https://gitlab.com/groups/gitlab-org/-/work_items/21832). - [Diagrams of the system topology](https://claude.ai/artifact/W2PnqJaNUAbtvDDuKB7wxe). - [A visualization of the whole execution trace](https://claude.ai/artifact/A1isD9HHoe3m3CwRv85ATK). - End-to-end integration branch: [!262838](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/262838) with ai-assist branch [`poc/ai-catalog-pull`](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/compare/main...poc/ai-catalog-pull). ## Internal cross-team comms channel :compass: :slack: We are using this channel for cross-team communication: https://gitlab.enterprise.slack.com/archives/C0BUB4J0DPD <details> <summary>Decisions from gitlab-org&23526 and the planning issue</summary> - Every reference pins an exact version. No support for the "latest". ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/627641#note_3805911237)) - Two rule sets: user permission and visibility hierarchy, for all catalog items, using the triggering user. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/627641#note_3785717552)) - Checks run again during execution against the container the flow executes from. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/628319#note_3892081394)) - A failed agent check blocks the execution and the error says which agent. Blocked MCP servers are dropped. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/628319)) - Twenty agent references per flow. Depth is flow, agent, the agent's MCP servers. ([source](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/256851)) - MCP tools of referenced agents are not in the first release. Built-in tools only. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/632032)) - DWS fetches the flow definition and its agents with the GET request, through Workhorse with the run's token. Nothing is resolved at preauthorization, and Workhorse doesn't change the start request. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/634265#note_4110946099), [proposal](https://gitlab.com/groups/gitlab-org/-/work_items/24006#note_4120524091)) - Rails resolves agent ids from the saved flow's dependency records, never from the request. A workflow with no saved Catalog item gets 422 from the GET request. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/628315)) - Workspace agents and AI Catalog agents are two lists in `CatalogItems` in DWS. We add the AI Catalog list and leave the workspace list to [!262949](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/262949). - Released at Experiment tier behind the Feature preview setting, the same shape as workspace agents. No new setting. ([source](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/259664)) - Still open for AppSec: which identity the checks use under a service account, and whether enabling a flow authorizes its whole chain. ([source](https://gitlab.com/gitlab-org/gitlab/-/work_items/628319#note_3902006973)) </details> ## How it works - Every entry point opens a session through Workhorse: the Duo CLI and IDE on `GET /api/v4/ai/duo_workflows/ws`, the CI job that the web UI starts on the same route, and headless callers on `POST /api/v4/ai/duo_workflows/workflows/:id/execute`. - Before DWS builds the flow, it calls `GET /api/v4/ai/duo_workflows/workflows/:id/catalog_items` ([#628315](https://gitlab.com/gitlab-org/gitlab/-/work_items/628315)) through Workhorse's HTTP request action, with the run's token ([ai-assist#3306](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/3306)). Rails has the workflow record, reads `workflow.ai_catalog_item_version`, and resolves the `include` entries from the dependency records with [`ReferenceResolver`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/catalog/flows/reference_resolver.rb) for the current user. It returns the flow definition and the agents. - With `ai_catalog_fetch_flow_config` on, the CI job gets a marker instead of the flow, and DWS checks the fetched flow against the token's `flow_config_digest` ([#634359](https://gitlab.com/gitlab-org/gitlab/-/work_items/634359)). No client sends a definition or an agent for a Catalog flow, and no environment variable holds them. - DWS binds one `AgentComponent` per fetched agent under the component that claims it ([ai-assist#2870](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/2870)). - The same GET request serves callers that need the items after the session has started: child sessions, `SubFlowComponent`, and the session page. ## First release A custom flow with Catalog subagents executes from the web UI, the Duo CLI, Agentic Chat, and headless callers, with built-in tools, at the Experiment tier, behind the Feature preview setting. The `ai_catalog_flow_agent_references` flag is on only for development. Every open child item is required for the beta release and has the ~Deliverable label. The child items are ordered most critical first. Follow-ups that the beta release does not need are in [gitlab-org&24639](https://gitlab.com/groups/gitlab-org/-/work_items/24639). ## Related efforts | Effort | What it does | Relation | |---|---|---| | [&75](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/code-suggestions/-/work_items/75), workspace agents GA (Alexander) | [!262949](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/262949) returns workspace agents for CI runs at preauthorization and attaches them in Workhorse. It's postponed, and Alexander plans to come back to it with the pull approach ([note](https://gitlab.com/gitlab-org/gitlab/-/work_items/634265#note_4110946099)). The Duo CLI checks local agents and lists them with `/agents`. DWS runs the deep check. | Same `CatalogItems` model in DWS. We do not build the workspace-agent side. | | [&55](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/code-suggestions/-/work_items/55) / [&80](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/code-suggestions/-/work_items/80), parent-child sessions (Mikołaj, Experiment in 19.6) | Session lifecycle as a contract: MRTR elicitation for human input, MCP Tasks semantics for child sessions, task handles on `start_flow` | A child session resolves its own `include` with the GET request ([ai-assist#3307](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/3307)). | | [#630476](https://gitlab.com/gitlab-org/gitlab/-/work_items/630476), bounded flow chaining (Igor, MRs 0–5 merged) | `parent_workflow_id`, `MAX_TRIGGER_DEPTH = 3`, `MAX_CHILD_WORKFLOWS_PER_PARENT = 5`, root human as the identity limit (`SA-n ∩ root human`) | Counts workflow records. [`ReferenceLimits`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/catalog/flows/reference_limits.rb) counts references. Each graph has its own limit. | ## Not in this epic Follow-ups that the beta release does not need, such as MCP tools of referenced agents and dependency information in the UI, are in [gitlab-org&24639](https://gitlab.com/groups/gitlab-org/-/work_items/24639). These items wait for other work and stay under [gitlab-org&21832](https://gitlab.com/groups/gitlab-org/-/work_items/21832): - The GET request from DWS for child sessions and mid-run claims ([ai-assist#3307](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/3307)) - Flows as subagents or components, in-band ([ai-assist#2749](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/work_items/2749)) and flows triggering flows, out-of-band (above). Both wait for parent-child sessions.