Model selection local dev problems

@nateweinshenker and I (@jessieay) recently spent several hours debugging a local GDK issue. He was trying to test model switching for Classic Chat to validate an MR but the requests were failing with the generic A9999 issue.

The next day, I went to test his MR and experienced the exact same thing.

Steps to reproduce (this happens on master too):

1. Run GDK in non-SaaS mode
2. Log in as instance admin
3. Try to use Classic Duo Chat
4. Chat responds with error: I'm sorry, I can't generate a response. Please try again. Error code: A9999

I have used Classic Chat locally a bunch in the past so this seemed new. I wasn't sure exactly what was going on but eventually figured out that this diff fixed the problem for me:

diff --git a/ee/app/models/ai/feature_setting.rb b/ee/app/models/ai/feature_setting.rb
index 13f16d64aa59..99fc16a9b0c0 100644
--- a/ee/app/models/ai/feature_setting.rb
+++ b/ee/app/models/ai/feature_setting.rb
@@ -129,9 +129,7 @@ def provider_title
     end

     def base_url
-      return Gitlab::AiGateway.cloud_connector_url if vendored?
-
-      Gitlab::AiGateway.url if self_hosted?
+      Gitlab::AiGateway.url
     end

     def compatible_self_hosted_models
diff --git a/ee/app/models/ai/model_selection/instance_model_selection_feature_setting.rb b/ee/app/models/ai/model_selection/instance_model_selection_feature_setting.rb
index 5438ee81c85c..f0d9785575c8 100644
--- a/ee/app/models/ai/model_selection/instance_model_selection_feature_setting.rb
+++ b/ee/app/models/ai/model_selection/instance_model_selection_feature_setting.rb
@@ -26,7 +26,7 @@ def model_selection_scope
       end

       def base_url
-        ::Gitlab::AiGateway.cloud_connector_url
+        ::Gitlab::AiGateway.url
       end

       def vendored?

Why does this fix the problem?

Locally, Gitlab::AiGateway.cloud_connector_url is the production AIGW (cloud.gitlab.com). I do not have a production license for my GDK. Therefore, when my instance tries to make a request to Gitlab::AiGateway.cloud_connector_url, it fails. When I try to make a request to my locally running AIGW instead (per diff above), it works.

Questions

1. Other than the diff above, are there workarounds for this? Setting CLOUD_CONNECTOR_BASE_URL does not work because we append /ai to that and this is an invalid path for a locally running AIGW.
2. Should the code be changed to allow for local development? I believe the root cause is that the models I am connecting to in my GDK are marked as vendored because I have not created any self-hosted model records. For customers, vendored models should go to the hosted AIGW ( Gitlab::AiGateway.cloud_connector_url). But folks running GitLab through GDK wouldn't be able to access production and they probably don't want to because it is easier to debug the system if all requests are made to local servers.