Respect user selected model when changing mid agentic chat
What does this MR do and why?
This MR fixes a bug introduced by the combination of !251007 (merged) and gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#2174.
The combination of allowing users to switch model mid chat (previously a new chat would be opened if you switched model), and storing the model used and reusing it in future requests (intended for default models which load balance between multiple providers, but also cache when on the same provider, therefore, we should only use the load balancing logic on the first request) introduced a bug where it would appear as if the user has switched model, but the chat will always use whichever model was selected in the first prompt.
This MR fixes this logic in 2 places:
ee/app/services/ai/duo_workflows/connection_config_service.rbOnly reuse the previous model if- The user has not passed a preference (using default).
- The
duo_workflow_provider_stickinessflag is enabled. - There is a previous model stored.
- The previous model included is in defaults.
ee/app/services/ai/duo_workflows/create_checkpoint_service.rbUpdate themodel_metadata_jsonon subsequent requests
References
gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#2901 (closed)
How to set up and validate locally
Ideally to be tested with multiple providers (AWS, Anthropic, Vertex). But if you do not have multiple providers set up, confirm switching between models of the same provider.
- Enable
Feature.enable(:duo_workflow_provider_stickiness) - Tail the logs
gdk tail duo-workflow-service | grep "Request to LLM"- This will display
Request to LLMwhich hasrequest_urltowards the end andRequest to LLM completewhich hasmodel_name
- This will display
- Start a chat using default
- Confirm a
claude-sonnet-4-6model is used, and is sent toaiplatform.googleapis.comorapi.anthropic.com
- Confirm a
- Confirm
Ai::DuoWorkflows::Workflow.last.model_metadata_jsonhas the expectednamefor the model (claude_sonnet_4_6_vertexfor vertex, orclaude_sonnet_4_6Anthropic) - Send a few more messages, and confirm the same endpoint is used (If you have Vertex and Anthropic models to confirm load balancing is not called for the default model).
- Switch any other model/provider
- Send a new message, and confirm the expected name/endpoint are used
- Confirm
Ai::DuoWorkflows::Workflow.last.model_metadata_jsonhas updated.
Confirm switching from a model included in defaults, to "default" does not change anything.
If a user is already using claude_sonnet_4_6_vertex vertex and they switch to default, it should still use the vertex model every time, and never switch to Anthropic (and vice versa)
If a user is using any other model that is not included in the defaults, and they switch to default, it should load balance between either of the defaults.
- Start a chat using either of the two models we default to
- Switch to default and confirm it continues using same model (through
Ai::DuoWorkflows::Workflow.last.model_metadata_jsonor the tail used above). - Switch to the other default provider and confirm the used model switches.
- Switch back to default, and confirm it continues using the model you just selected.
- Switch to any other model and confirm switch again.
- Switch back to defaults, and it should use either of the two defaults.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
