<!-- Bug template: https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Bug.md --> ### Summary Replying to or mentioning `@GitLabDuo` in merge request discussions (non-agentic Duo Code Review) fails with **"I'm sorry, I can't generate a response. Please try again."** on air-gapped GitLab Duo Self-Hosted instances with offline cloud licenses and the Duo Enterprise add-on (no Agent Platform add-on). Duo Chat and initial `@GitLabDuo` reviews work correctly on the same instances. Confirmed on **19.2.1**; the same symptom was reported on **18.11.7** (mechanism on that version not verified). ### Root cause (verified on `19-2-stable-ee`) 1. `MergeRequests::DuoCodeReviewChatWorker` executes mention replies via `Gitlab::Llm::Completions::Chat` with `is_duo_code_review: true`. 1. `ee/lib/gitlab/llm/completions/chat.rb` sets the unit primitive to `:duo_chat` for this path (regular Chat uses `:duo_classic_chat`). The code carries `# TODO: revert to duo_classic_chat, see gitlab-org/gitlab#585237`. 1. `CloudConnector::Tokens.get` (`ee/lib/cloud_connector/tokens.rb`) self-signs the AI Gateway token **only** when the unit primitive maps to a self-hosted feature setting. Per `ee/lib/gitlab/ai/feature_settings/feature_metadata.yml`, `duo_classic_chat` maps to the **General Chat** feature, but `duo_chat` maps to `duo_agent_platform_agentic_chat` (**Agentic Chat**). 1. Offline cloud licenses without an Agent Platform add-on purchase cannot configure that feature setting: `GlobalPolicy#dap_self_hosted?` (`ee/app/policies/ee/global_policy.rb`) prevents `read/update_dap_self_hosted_model`, so the setting is hidden in the UI and blocked in GraphQL. 1. The code therefore falls back to `cloud_connector_token` (CustomersDot-synced), which does not exist on air-gapped instances. Rails sends a blank bearer token; the AI Gateway fails JWT decode before reading the issuer claim. ### Steps to reproduce 1. docker compose environment: `gitlab/gitlab-ee:19.2.1-ee.0` + `gitlab/model-gateway:self-hosted-v19.2.1-ee` on an isolated network. Offline cloud license (Ultimate) with Duo Enterprise. Self-hosted model assigned to General Chat and Code Review. `ai_gateway_url` pointing at the local gateway. 1. Ensure `AIGW_AUTH__BYPASS_EXTERNAL` is unset/false on the gateway (setting it to `true` masks this bug entirely — this is why earlier reproduction attempts passed) and `CLOUD_CONNECTOR_SELF_SIGN_TOKENS` is unset on GitLab. 1. Confirm no synced CustomersDot token: `CloudConnector::Tokens.cloud_connector_token.present?` → `false`. 1. Verify Duo Chat and an initial `@GitLabDuo` review work. 1. Reply to any `@GitLabDuo` review comment or mention `@GitLabDuo` in an MR discussion. ### What is the current *bug* behavior? The reply fails with "I'm sorry, I can't generate a response. Please try again." AI Gateway log shows: ``` path: /v2/chat/agent, status_code: 401, auth_error_details: "Forbidden by auth provider", token_issuer: "" ``` preceded by a `cloud_connector` logger entry with `Not enough segments`. ### What is the expected *correct* behavior? Mention replies authenticate the same way as every other self-hosted Duo request on the instance (locally signed token) and succeed. ### Environment - GitLab version: 19.2.1 (confirmed); same symptom reported on 18.11.7 (mechanism not verified) - Deployment type: GitLab Duo Self-Hosted (Self-Managed, air-gapped) - License: Offline cloud license (Ultimate) with Duo Enterprise add-on; no Agent Platform add-on - AI Gateway: `gitlab/model-gateway:self-hosted-v19.2.1-ee` ### Impact - **Primarily affected:** air-gapped/offline-license Duo Enterprise customers without the Agent Platform add-on. Structurally broken; no supported remediation (the enabling feature setting is license-gated away from them). - **Secondarily affected:** online-license Duo Self-Hosted instances whose AI Gateway cannot reach customers.gitlab.com (JWKS fetch fails; `Signature verification failed` variant of the same 401), or whose CustomersDot token sync is stale. - **Not affected:** online-license instances whose gateway can reach CustomersDot, and any instance with `duo_agent_platform_agentic_chat` assigned to a self-hosted model. - **Latent inconsistency even where it works:** mention replies are the only self-hosted-Duo path authenticating via CustomersDot attestation instead of instance-signed tokens. ### Severity rationale ~"severity::2": feature is completely non-functional for the affected customer class and the only workarounds are unsupported: one is documented as not-for-production, the other bypasses a license-enforcement policy. Downgrade to ~"severity::3" if triage considers the unsupported workarounds sufficient. ### Possible fixes Revert the mention-path unit primitive to `duo_classic_chat` per the existing code TODO, or derive token issuance for `is_duo_code_review` requests from the General Chat feature setting. ### Validated workarounds (both unsupported) - `CLOUD_CONNECTOR_SELF_SIGN_TOKENS=1` on the GitLab instance (restores function; documented as not-for-production in [GitLab Duo Self-Hosted troubleshooting docs](https://docs.gitlab.com/administration/gitlab_duo_self_hosted/troubleshooting/)). ### Related (not duplicates) - #585237 — the mention messaging adapter feature; the root-cause code carries a TODO referencing it - !237658 — MR that introduced the unit primitive override - gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#1783 — same log fingerprint (`Forbidden by auth provider`, `token_issuer: ""`, `Not enough segments`), different endpoint (`/v1/prompts/model_configuration/check`) and root cause