Skip to content

chore: remove per-message prompt limits for Duo Chat

What does this merge request do and why?

Removes the per-message limit for chat messages, to allow the limits to be handled by the LLM provider.

Based on this comment.

How to set up and validate locally

You can apply this patch to the monolith to send an absurdly long message.

Subject: [PATCH] Test character limit
---
Index: ee/lib/gitlab/duo/chat/react_executor.rb
<+>UTF-8
===================================================================
diff --git a/ee/lib/gitlab/duo/chat/react_executor.rb b/ee/lib/gitlab/duo/chat/react_executor.rb
--- a/ee/lib/gitlab/duo/chat/react_executor.rb
+++ b/ee/lib/gitlab/duo/chat/react_executor.rb
@@ -26,7 +26,7 @@
         # @param [ResponseService] response_handler - Handles returning the response to the client
         # @param [ResponseService] stream_response_handler - Handles streaming chunks to the client
         def initialize(user_input:, tools:, context:, response_handler:, stream_response_handler: nil)
-          @user_input = user_input
+          @user_input = 'a' * 1000000
           @tools = tools
           @context = context
           @iterations = 0

I did this, and on main, we just get an A9999 error.

image

But on this branch, you get a proper message that says that the prompt is too long.

image

Merge request checklist

- [ ] Tests added for new functionality. If not, please raise an issue to follow up.

- [ ] Documentation added/updated, if needed.

Edited by Lesley Razzaghian

Merge request reports

Loading