Skip to content

Increase message limit to 100

Jessie Young requested to merge jy-message-list-length into main

What does this merge request do and why?

  • Previous limit was 10, which was set via this MR
  • In the monolith, we save up to 50 Duo Chat messages.
  • Message history is sent with each request to the AI Gateway. We send the user prompt and the Duo Chat answer for each item in the history.
  • We either need to change the amount of history we send from the monolith or the message limit in the Gateway.
  • More discussion on history / context window limits here: gitlab-org/gitlab#444629 (comment 1838552224)

How to set up and validate locally

  1. Check out to this merge request's branch.
  2. Check out the branch for the monolith on this branch.
  3. Make 5+ Duo Chat requests
  4. Can make a 6th Duo chat request.
  5. On the main branch of the AI Gateway, the 6th request includes the following in the response:
"ctx\\\":{\\\"field_type\\\":\\\"List\\\",\\\"max_length\\\":10,\\\"actual_length\\\":11},\\\"url\\\":\\\"https://errors.pydantic.dev/2.6/v/too_long\\\"

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Jessie Young

Merge request reports