<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Label this issue](https://contributors.gitlab.com/manage-issue?action=label&projectId=278964&issueIid=599841) - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=599841) </details> <!--IssueSummary end--> > ⚠️ **This issue was drafted with the help of an AI assistant** based on the discussions in !235204. It captures the current understanding but has not yet been independently sanity-checked. Please review the proposed phases, code pointers, and open questions critically before acting on them - some details may be inaccurate or out of date once !235204 merges and rebases. ## Problem The string `developer/v1` (stored as `Ai::DuoWorkflows::Workflow#workflow_definition` and `Ai::Catalog::FoundationalFlow#foundational_flow_reference`) is currently overloaded with three distinct meanings: - **Stable feature identity** used as a billing/analytics key (`label` on internal events like `start_duo_workflow_execution`, the `workflow_definition` additional property on `duo_workflow_workload_completed`, audit event `target_details`, and the `WORKFLOW_EVENTS` mapping in [`ee/lib/api/ai/duo_workflows/workflows.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/ai/duo_workflows/workflows.rb)). - **Engine routing key** for the AI Gateway Flow Registry. Currently derived by splitting the combined string on `/` in [`Ai::DuoWorkflows::FoundationalFlowStartParamsResolver`](https://gitlab.com/gitlab-org/gitlab/-/blob/ts/flow-versioning-support/ee/lib/ai/duo_workflows/foundational_flow_start_params_resolver.rb) (introduced in !235204). - **Config version**, where the `/v1` suffix is sometimes interpreted as a SemVer-like indicator even though it's actually the registry schema version. The new explicit `flow_version` attribute (e.g. `2.0.0` for `developer/v1`) introduced in !235204 makes that ambiguity concrete. !235204 introduced explicit `flow_config_id`, `flow_config_schema_version`, and `flow_version` params for the engine side, but left the billing/feature identity coupled to the same combined string. This issue tracks the follow-up cleanup. Relevant discussion: [thread starting here](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235204#note_3333134409) (Luke's "stable ID + SemVer + schema version" framing), [Thomas' response](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235204#note_3333436628) outlining the four-property model, [Mikolaj's `billing_id` / `feature_id` proposal](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235204#note_3333890030), and [Thomas' observation](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235204#note_3337942785) that the flow-triggering surface area has grown organically and would benefit from a holistic look. ## Goals - Introduce a **stable feature ID** that billing/analytics can rely on, decoupled from engine routing decisions. - Stop deriving `flow_config_id` / `schema_version` by string-splitting `foundational_flow_reference` (currently in [`FoundationalFlowStartParamsResolver#call`](https://gitlab.com/gitlab-org/gitlab/-/blob/ts/flow-versioning-support/ee/lib/ai/duo_workflows/foundational_flow_start_params_resolver.rb)). - Stay green-deployable: no DB migrations, no ClickHouse changes, no broken historical analytics in the first step. ## Out of scope - Renaming the `duo_workflows_workflows.workflow_definition` DB column (high cost, low marginal benefit). The column keeps its current name and values; only its *interpretation* by emitters shifts. - Renaming the public REST param `workflow_definition` (documented in [`doc/api/duo_agent_platform_flows.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/api/duo_agent_platform_flows.md)) or the GraphQL field `workflowDefinition` ([`ee/app/graphql/types/ai/duo_workflows/workflow_type.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/graphql/types/ai/duo_workflows/workflow_type.rb)) - separate API contract change. - A holistic refactor of the flow-triggering paths (`Ai::DuoWorkflows::CreateAndStartWorkflowService`, `Ai::Catalog::ExecuteWorkflowService`, `Ai::Catalog::Flows::ExecuteService`, `Ai::Messaging::TriggerFlowService`, vulnerability triggers). To be tracked separately. ## Proposed approach (phased) ### Phase 1 - explicit attributes on `FoundationalFlow` (no DB change, no external contract change) 1. Add a `feature_id` attribute to [`Ai::Catalog::FoundationalFlow`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/ai/catalog/foundational_flow.rb) with values that visibly differ from `flow_config_id`. Proposed: `duo_developer`, `duo_fix_pipeline`, `duo_code_review`, `duo_sast_fp_detection`, `duo_secrets_fp_detection`, `duo_resolve_sast_vulnerability`, `duo_convert_to_gl_ci`. The `duo_*` prefix follows the precedent of [`SelfHostedBillingTracker::FEATURE_QUALIFIED_NAME`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ai/code_suggestions/self_hosted_billing_tracker.rb). 2. Add explicit `flow_config_id` and `flow_config_schema_version` attributes on `FoundationalFlow` (default values per flow, no longer derived by parsing `foundational_flow_reference`). 3. Update [`FoundationalFlowStartParamsResolver`](https://gitlab.com/gitlab-org/gitlab/-/blob/ts/flow-versioning-support/ee/lib/ai/duo_workflows/foundational_flow_start_params_resolver.rb) to read those explicit attributes directly rather than splitting on `/`, and to also return `feature_id` in its result hash. 4. Address [Luke's open question about `container` typing](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235204#note_3337082040): the resolver currently accepts either a `Project` or `Namespace` implicitly. Either rename the arg, add a type guard, or accept both explicitly with documented behavior. ### Phase 2 - dual-emit on billing/analytics events (event schema additions, value preserved for back-compat) 1. Add `feature_id` to `additional_properties` in the relevant event YAMLs: [`start_duo_workflow_execution`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/events/start_duo_workflow_execution.yml), [`finish_duo_workflow_execution`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/events/finish_duo_workflow_execution.yml), [`retry_duo_workflow_execution`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/events/retry_duo_workflow_execution.yml), [`duo_workflow_workload_completed`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/events/duo_workflow_workload_completed.yml), and the `agent_platform_session_*` events. 2. In [`WorkflowEventTracking#workflow_tracking_properties`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/duo_workflows/concerns/workflow_event_tracking.rb) and [`WorkloadMetrics#track_workload_completion_metrics`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/duo_workflows/concerns/workload_metrics.rb), resolve `feature_id` via `FoundationalFlow.find_by_reference(workflow.workflow_definition)&.feature_id` and pass it through. Keep `label: workflow.workflow_definition` for back-compat. 3. For custom flows (`workflow_definition == "ai_catalog_agent"`) and the legacy default (`workflow_definition == "software_development"`), emit shared `feature_id` values (proposal: `duo_custom_flow` and `duo_software_development`). 4. Update [`API::Ai::DuoWorkflows::Workflows#track_event`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/ai/duo_workflows/workflows.rb) (vulnerability flow billing trigger) to look up `feature_id` rather than keying its `WORKFLOW_EVENTS` hash by `foundational_flow_reference`. 5. Migrate dashboards/queries to prefer `feature_id` over `label`. ### Phase 3 - clean up remaining hardcoded references (low-risk hygiene) 1. [`Ai::Messaging::TriggerFlowService::WORKFLOW_DEFINITION_REF = 'developer/v1'`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/messaging/trigger_flow_service.rb) - resolve via `FoundationalFlow` lookup instead of a hardcoded string. 2. [`Ai::FoundationalChatAgent.reference_from_workflow_definition`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/ai/foundational_chat_agent.rb) - audit whether chat agents should also gain a `feature_id` rather than the same split convention. 3. [`Vulnerabilities::TriggeredWorkflow#workflow_name`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/vulnerabilities/triggered_workflow.rb) - this is already a separate enum (`sast_fp_detection`, `resolve_sast_vulnerability`, `secrets_fp_detection`). Useful precedent, confirm alignment with the new `feature_id` values. ### Phase 4 (optional, future) - persist `feature_id` on the workflow row Only if querying/filtering by stable feature ID becomes necessary. Requires column add + backfill + ClickHouse siphon update + GraphQL field. Defer until Phase 2 dashboards confirm the property works. ## Open questions - **AI Gateway symmetry.** Thomas [noted](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/235204#note_3333436628) that the AI Gateway likely emits events keyed on `developer/v1`-style strings too. Coordinate with the [ai-assist](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist) team to align on the same `feature_id` concept across both services. Without this, Phase 2 dashboards would still see split series from the gateway side. - **Feature flag overrides and billing.** Today the `duo_developer_next_unstable` override in [`FoundationalFlowStartParamsResolver`](https://gitlab.com/gitlab-org/gitlab/-/blob/ts/flow-versioning-support/ee/lib/ai/duo_workflows/foundational_flow_start_params_resolver.rb) swaps the reference from `developer/v1` to `developer_unstable/experimental`, which also implicitly swaps the billing label. Should `feature_id` stay `duo_developer` regardless of the override, or should the unstable variant have its own `feature_id`? - **Custom flow / external agent billing buckets.** Confirm with product that one shared `feature_id` per flow type (one for custom, one for external agents) is sufficient, vs needing per-item granularity. Per current understanding, no sub-classification needed. - **`ai_catalog_item_version_id` scope.** It's a GitLab-side audit/billing/UI concept and is **not** sent to the Duo Workflow Service. Confirm it stays that way and is not surfaced on engine env vars. ## Downstream usage inventory (informational - to be verified during implementation) Places where the combined `workflow_definition` string is read today: - [`Ai::DuoWorkflows::Workflow#workflow_definition`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/ai/duo_workflows/workflow.rb) - persisted text column (default `'software_development'`, 255-char check constraint), replicated to ClickHouse `siphon_duo_workflows_workflows`. - [`Ai::DuoWorkflows::Concerns::WorkflowEventTracking#workflow_tracking_properties`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/duo_workflows/concerns/workflow_event_tracking.rb) - Snowplow `label`. - [`Ai::DuoWorkflows::Concerns::WorkloadMetrics#track_workload_completion_metrics`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/duo_workflows/concerns/workload_metrics.rb) - Snowplow `workflow_definition` additional property. - [`Ai::DuoWorkflows::UpdateWorkflowStatusService#audit_event_for_status_change`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/duo_workflows/update_workflow_status_service.rb) - audit event `target_details`. - [`API::Ai::DuoWorkflows::Workflows#track_event` and `WORKFLOW_EVENTS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/ai/duo_workflows/workflows.rb) - vulnerability flow billing. - [`Vulnerabilities::TriggeredWorkflow#workflow_name`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/vulnerabilities/triggered_workflow.rb) - already a separate enum. - [`Ai::Catalog::ExecuteWorkflowService#determine_workflow_definition`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/catalog/execute_workflow_service.rb) - sets `'ai_catalog_agent'` for custom flows. - [`Ai::Catalog::Flows::ExecuteService#fetch_flow_definition`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/catalog/flows/execute_service.rb) - foundational vs custom branching. - [`Ai::Messaging::TriggerFlowService::WORKFLOW_DEFINITION_REF`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/ai/messaging/trigger_flow_service.rb) - hardcoded `'developer/v1'`. - [`Ai::FoundationalChatAgent.reference_from_workflow_definition`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/ai/foundational_chat_agent.rb) - chat agent split logic. - GraphQL: [`DuoWorkflow.workflowDefinition` field](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/graphql/types/ai/duo_workflows/workflow_type.rb), [`create` mutation arg](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/graphql/mutations/ai/duo_workflows/create.rb), [`workflows` resolver `type` filter](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/graphql/resolvers/ai/duo_workflows/workflows_resolver.rb). - REST: `workflow_definition` POST param at `/api/v4/ai/duo_workflows/workflows` ([docs](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/api/duo_agent_platform_flows.md)). - Background migration `backfill_service_account_id_on_duo_workflows_workflows` (uses workflow_definition + ai_catalog_item_version_id). - [`scripts/dap_foundational_flows_release_notes.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/dap_foundational_flows_release_notes.rb) (parses `foundational_flow.rb` for references). ## Related - !235204 (Support flow versioning for foundational flows) - &18785 (Flow versioning epic)