Increase message limit to 100
What does this merge request do and why?
- Previous limit was 10, which was set via this MR
- In the monolith, we save up to 50 Duo Chat messages.
- Message history is sent with each request to the AI Gateway. We send the user prompt and the Duo Chat answer for each item in the history.
- We either need to change the amount of history we send from the monolith or the message limit in the Gateway.
- More discussion on history / context window limits here: gitlab-org/gitlab#444629 (comment 1838552224)
How to set up and validate locally
- Check out to this merge request's branch.
- Check out the branch for the monolith on this branch.
- Make 5+ Duo Chat requests
- Can make a 6th Duo chat request.
- On the
main
branch of the AI Gateway, the 6th request includes the following in the response:
"ctx\\\":{\\\"field_type\\\":\\\"List\\\",\\\"max_length\\\":10,\\\"actual_length\\\":11},\\\"url\\\":\\\"https://errors.pydantic.dev/2.6/v/too_long\\\"
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Jessie Young
Merge request reports
Activity
Please register or sign in to reply