Skip to content

Draft: Fail tool calls if ended due to max_tokens

Dylan Griffithrequested to merge
do-not-execute-tools-if-stop-reason-max-tokens into main

What does this merge request do and why?

Related to !3459 (merged) . Do not run a tool if stop_reason == "max_tokens" as this indicates that the LLM did not finish constructing a valid response. Per !3459 (comment 2787259164)

When a tool-use is streamed, we send NewCheckpoint with the snapshot of the state done before the tool-use started being generated.

Combined with fine-grained-tool-streaming-2025-05-14 beta feature:

  • It'll help to keep the channel busy while the tool-use is generated
  • It the first step towards tool-use message streaming

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
  • If this change requires executor implementation: verified that issues/MRs exist for both Go executor and Node executor or confirmed that changes are backward-compatible and don't break existing executor functionality.

Merge request reports