Skip to content

Stream ReAct agent responses

Alexander Chueshev requested to merge ac/add-streaming-react-agent into main

What does this merge request do and why?

This MR adds streaming support to the ReAct agent implemented in !720 (merged). More details in #468 (closed).

How to set up and validate locally

  1. Request with the empty scratchpad:

    curl -X 'POST' \
   'http://localhost:5052/v2/chat/agent' \
   -H 'accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{
       "prompt": "What'\''s the title of this issue?",
       "options": {
         "chat_history": "",
         "agent_scratchpad": {
           "agent_type": "react",
           "steps": []
         }
       }
   }'

  2. Request with the existing scratchpad:

    curl -X 'POST' \
 'http://localhost:5052/v2/chat/agent' \
 -H 'accept: application/json' \
 -H 'Content-Type: application/json' \
 -d '{
      "prompt": "What'\''s the title of this issue?",
      "options": {
        "chat_history": "",
        "agent_scratchpad": {
          "agent_type": "react",
          "steps": [
            {
              "thought": "To get the title and other details of this issue, I need to use the \"issue_reader\" tool.",
              "tool": "issue_reader",
              "tool_input": "What'\''s the title of this issue?",
              "observation": "here is the issue about implementing a PoC for ...."
            }
          ]
        }
      }
   }'

  3. Request with the existing chat history (either string or a list of strings):

    curl -X 'POST' \
  'http://localhost:5052/v2/chat/agent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
        "prompt": "What'\''s your name?",
        "options": {
          "chat_history": ["User: What'\''s your name?", "AI: My name is Skynet."],
          "agent_scratchpad": {
            "agent_type": "react",
            "steps": []
          }
        }
   }'

  4. Request with the pre-defined resource (issue or epic):

    curl -X 'POST' \
  'http://localhost:5052/v2/chat/agent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
        "prompt": "What'\''s the title of this issue?",
        "options": {           
           "chat_history": "",
           "agent_scratchpad": {
             "agent_type": "react",
             "steps": []
           },
          "resource": {
            "type": "issue",
            "content": "title: PoC ReAct, description: some description"
          }
        }
    }'

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Alexander Chueshev

Merge request reports