Skip to content

feat: add tool response truncation

What does this merge request do and why?

This MR is to truncate tool output into 5kib if it exceeds 100kib size as proposed in issue gitlab-org/gitlab#571707. And add instruction to tell LLM the output has been truncated and suggestion of alternative approaches.

At the same time, we are asking advice from Anthropic team to give inputs of above configuration.

How to set up and validate locally

  1. checkout mr
  2. ask to summarize a big csv or text file
  3. it will not throw error and the response should have a note that it is based on incomplete tool output

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
  • If this change requires executor implementation: verified that issues/MRs exist for both Go executor and Node executor or confirmed that changes are backward-compatible and don't break existing executor functionality.
Edited by Junming Huang

Merge request reports

Loading