## Summary Roll out [the feature](https://gitlab.com/gitlab-org/gitlab/-/work_items/627566) currently behind the `dap_schema_auto_tool_choice` feature flag. - DRI: @kinsingh - Team Slack channel: `#g_ai_coding` > [!note] > Process and guidance live in the docs, this issue is just the commands and a place to track the rollout. > "Rolling out" means incrementally enabling the flag on GitLab.com to validate stability, it is not the same as releasing the feature, which happens when the flag is removed. > [Feature flag controls](https://docs.gitlab.com/development/feature_flags/controls/) · [Feature flag lifecycle](https://handbook.gitlab.com/handbook/product-development/how-we-work/product-development-flow/feature-flag-lifecycle/#feature-flag-lifecycle) ## What could go wrong? - Scope is wider than one flow. The flag is read inside `AgentComponent`, so it changes tool binding for every schema-bound agent the actor touches, across nine flows. Five are GA: `fix_pipeline`, `sast_fp_detection`, `secrets_fp_detection`, `convert_to_gl_ci`, `resolve_sast_vulnerability`. - The sharpest risk is components declaring `toolset: []`, where the response schema tool is the only tool bound, so `"any"` makes them deterministic one-shot structured extractors. These are `fix_pipeline_decide_fix`, `readiness_score`'s `rubric` and `coverage`, and `sast_fp_detection`'s `validate_sast_vulnerability_component`. - Under `"auto"` a prose reply from one of those components routes back to the agent instead of ending the loop, and none of them set `max_cycles`, so they inherit the 280 default. That is a cost and latency regression first, and an `AgentStuckError` at the end. - Those prompts were written under a forced tool choice and never instruct the model to call the tool. For example `fix_pipeline_decide_fix` says "Set the `decision` field". - Every text-only deliberation turn is a billed model call, so cost per flow run can rise even when the run succeeds. - Any regression is contained to flagged actors. Disabling the flag returns them to `"any"` immediately, and no state persists across the change. Detection: Duo Workflow Service logs for `AgentStuckError` and the wrap-up warning "Agent ignored wrap-up instruction and made non-final tool calls", plus cycle counts on flows that previously completed in a handful of cycles, and feedback in `#g_ai_coding`. ## Rollout Run all production `/chatops` in [`#production`](https://gitlab.slack.com/archives/C101F3796) and cross-post the results to `#g_ai_coding`. Background: [incremental rollout process](https://docs.gitlab.com/development/feature_flags/controls/#process), [feature actors](https://docs.gitlab.com/development/feature_flags/#feature-actors). Before enabling for anyone, confirm both prerequisites are deployed: 1. Monolith MR https://gitlab.com/gitlab-org/gitlab/-/merge_requests/254206 deployed to production. Without this the flag does not exist and is never sent to the gateway. 1. ai-assist MR https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/6734 deployed to the GitLab-managed AI Gateway and Duo Workflow Service fleet. Nothing reads the flag until this is deployed, so enabling it earlier is inert rather than harmful. **Non-production** ``` /chatops gitlab run feature set dap_schema_auto_tool_choice true --dev --pre --staging --staging-ref ``` **Production** This rollout is actor based, not percentage based. Step 1: enable for named Agent Platform and Code Review team members. Step 2: enable for all internal GitLab team members. ``` /chatops gitlab run feature set --user=<username> dap_schema_auto_tool_choice true ``` ``` /chatops gitlab run feature set --group=gitlab-com dap_schema_auto_tool_choice true ``` A global percentage rollout is not planned yet. See "Before global rollout" below. ## Before global rollout Confirm the relevant gotchas before going to 100%, see [enabling a feature for GitLab.com](https://docs.gitlab.com/development/feature_flags/controls/#enabling-a-feature-for-gitlabcom): - [ ] [Docs + version history](https://docs.gitlab.com/development/documentation/feature_flags/) updated - [ ] [Breaking changes](https://docs.gitlab.com/development/documentation/release_notes/#deprecations-removals-and-breaking-changes) announced, if any - [ ] [Change management issue](https://handbook.gitlab.com/handbook/engineering/infrastructure-platforms/change-management/#feature-flags-and-the-change-management-process) opened, if required - [ ] [External API consumers](https://docs.gitlab.com/development/feature_flags/#do-not-use-feature-flags-in-external-api-consumers) handled with a fail-open mechanism, if applicable - [ ] The four `toolset: []` components above verified under `"auto"`, either by a dogfood run or by reprompting them to name the schema tool explicitly - [ ] Owning groups for the five GA flows signed off, since the flag changes their agents too ## Cleanup The flag exists to derisk making `"auto"` the unconditional binding in `AgentComponent`. It is removed once that binding is the default, at which point the flag read and the `AgentNode` tool choice argument that carries it both go away. Remove the flag and its YAML definition from the codebase, then: ``` /chatops gitlab run release check <merge-request-url> <milestone> /chatops gitlab run feature delete dap_schema_auto_tool_choice --dev --pre --staging --staging-ref --production ``` ## Rollback ``` /chatops gitlab run feature set dap_schema_auto_tool_choice false # production /chatops gitlab run feature set dap_schema_auto_tool_choice false --dev --pre --staging --staging-ref # non-production /chatops gitlab run feature delete dap_schema_auto_tool_choice --dev --pre --staging --staging-ref --production # remove entirely ```