Sign chat tool requests from the resolved feature setting

Kaveh Nejadrequested to merge
621674-sign-tool-requests-from-feature-setting into master

What does this MR do and why?

Follow up from !254156 (comment 3863706816). !254156 (merged) which fixes a bug identified in #621674 (closed) where GitLab Duo self-hosted Duo Enterprise with offline licenses instances can request reviews from GitlabDuo, but if they attempt to tag duo in a comment, Duo fails to respond. !254156 (merged) fixes the bug for simple responses that do not trigger tool calls. However !254156 (comment 3863706816) identified comments which will cause Duo to use a tool call can still fail as the authentication uses a separate route.

This MR updates the tool action path of the GtilabDuo @ mention logic to pass in the feature_setting when creating Gitlab::AiGateway.headers in a similar was as !254156 (diffs) did for the non tool call path.

References

Bug issue: #621674 (closed)

Fix for non tool messages: !254156 (merged)

Screenshots or screen recordings

Before After
image.png image.png

How to set up and validate locally

Environment set up

Prerequisites:

  1. AIGW set up locally
  2. gitlab runner
  3. Self hosted model, or a provider API key (Anthropic)

Set up steps

GDK in self-managed mode
  1. In gdk.yml: env: GITLAB_SIMULATE_SAAS: "0"
  2. Make sure CLOUD_CONNECTOR_SELF_SIGN_TOKENS appears in neither gdk.yml env: nor env.runit.
AI Gateway auth

In gitlab-ai-gateway/.env:

  • AIGW_CUSTOM_MODELS__ENABLED=true
  • AIGW_GITLAB_URL=http://:3000
  • AIGW_AUTH__BYPASS_EXTERNAL — unset or false
  • AIGW_AUTH__BYPASS_JWT_SIGNATURE — unset/false
  • AIGW_LOGGING__ENABLE_REQUEST_LOGGING=true — needed for the auth_error_details / token_issuer lines

gdk restart

License + add-ons (Duo Enterprise, no Agent Platform)

From the gitlab dir.

DEVELOPMENT_AI_GATEWAY_URL=http://localhost:5052 bundle exec rake "gitlab:duo:setup[duo_enterprise]"

bundle exec rake gitlab:duo:seed_self_hosted_models

Model set up

Option 1. Using Anthropic (recommended)

In http://gdk.test:8080/admin/gitlab_duo/model_selection/models/new use the following variables

Field Value
Name anthropic-opus-5 - self hosted - pointing to API - Or any other name
Platform API
Family Claude
Ednpoint https://api.anthropic.com
Identifier anthropic/claude-opus-5
API key your sk-ant-... key

Option 2. Locally running self hosted (not recommended)

I was also able to test this with a ollama-qwen25-7b (Llama 3) model.

Ollama set up and issues It required the following diff 
kaveh@kvejad--20251026-73SB4:~/gdk/gitlab-ai-gateway$ git diff
diff --git a/ai_gateway/prompts/definitions/chat/merge_request_reader/llama3/1.0.0.yml b/ai_gateway/prompts/definitions/chat/merge_request_reader/llama3/1.0.0.yml
index 5b8d5c903c..dc6d2900db 100644
--- a/ai_gateway/prompts/definitions/chat/merge_request_reader/llama3/1.0.0.yml
+++ b/ai_gateway/prompts/definitions/chat/merge_request_reader/llama3/1.0.0.yml
@@ -14,3 +14,5 @@ prompt_template:
     {% include 'chat/merge_request_reader/assistant/1.0.0.jinja' %}
 params:
   timeout: 60
+  stop:
+    - "```"
diff --git a/ai_gateway/prompts/definitions/chat/react/llama3/1.0.0.yml b/ai_gateway/prompts/definitions/chat/react/llama3/1.0.0.yml
index d0fa4abeca..409d486dd5 100644
--- a/ai_gateway/prompts/definitions/chat/react/llama3/1.0.0.yml
+++ b/ai_gateway/prompts/definitions/chat/react/llama3/1.0.0.yml
@@ -13,6 +13,6 @@ prompt_template:
   assistant: |
     {% include 'chat/react/assistant/1.0.0.jinja' %}
 params:
-  timeout: 30
+  timeout: 200
   stop:
     - "Observation:"
  1. Follow https://docs.gitlab.com/development/ai_features/local_models/ set up steps
  2. Update http://gdk.test:8080/admin/gitlab_duo/model_selection/models/new with the new model

Main issues are:

  1. It is very slow
  2. It cannot handle much context, and is likely to fail for other reasons
Feature settings

At /admin/gitlab_duo/model_selection:

  • Assign a self-hosted model to General Chat (duo_chat feature) and Non-Agentic Code Review (review_merge_request),.
  • Leave Agentic Chat (duo_agent_platform_agentic_chat) on the vendored default. That gap is exactly what stops :duo_chat from self-signing.
Simulate the air-gap (no CustomersDot token)

gdk rails c

CloudConnector::ServiceAccessToken.delete_all
CloudConnector::Tokens.cloud_connector_token.present? # => false
Force the non-agentic mention path

gdk rails c

gitlab:duo:setup enables all foundational flows, which routes mentions to the DAP handlers instead of the buggy chat path. On the test group/project, disable the code_review/v1 foundational flow (or duo_foundational_flows_enabled), then confirm:

project = Project.find_by_full_path('gitlab-duo/test')
user = User.find_by_username('root')

project.project_setting.update!(duo_foundational_flows_enabled: false)

Gitlab::Duo::CodeReview::ModeResolver.new(user: user, container: project).mode # => :classic
Verify preconditions, then reproduce
Ai::FeatureSetting.feature_for_unit_primitive(:duo_classic_chat)&.self_hosted? # => true 
Ai::FeatureSetting.feature_for_unit_primitive(:duo_chat)&.self_hosted? # => nil/false ← bug trigger 
CloudConnector::Tokens.get(unit_primitive: :duo_chat, resource: :instance) # => nil
CloudConnector::Tokens.get(unit_primitive: :duo_classic_chat, resource: :instance) # => JWT
Disable comment temperature and ai_prompts_v2
Feature.disable(:comment_temperature)
Feature.disable(:ai_prompts_v2) # https://gitlab.slack.com/archives/C06LWENL58F/p1789476326057429

Test

  1. Assign GitLabDuo as a reviewer of an MR
    1. Confirm Duo is able to review the MR
  2. Tag Duo in a comment that requires any tool usage
    1. @GitlabDuo give me a summary of this MR
  3. Without the changes in this MR, the request should fail, with the change, you should see a response.
  4. Reset the gdk and env settings
    1. Undo changes in the two files
    2. gdk reconfigure gdk restart
    3. Retry command and confirm it still passes (we didn't break the other set ups)
  5. Update /admin/gitlab_duo/model_selection to point to a non self managed model
    1. Retry command and confirm it still passes (we didn't break the other set ups)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #621674 (closed)

Edited by Kaveh Nejad

Merge request reports