Parse unformatted answers from LLM

Summary

This bug was discovered first for follow up questions when sometimes the model returns unformatted response. As AIGW chat parser works only in streaming mode, if answer doesn't have Final Answer: part, it just returns an empty string.

Steps to reproduce

1. Go to Duo Chat and clean history
2. Ask "Java", wait for reply when Chat describe Java lang
3. Ask "write hello world", wait till Chat write a script in different languages
4. Ask "in java"

Alternatively, change instructions in the V2 Chat prompt in AI GW to not return Final Answer:.

What is the current bug behavior?

Chat replies:

I'm sorry, I couldn't respond in time. Please try a more specific request or enter /clear to start a new chat.

What is the expected correct behavior?

Hello world in Java

Relevant logs and/or screenshots

Based on logs, for the last question LLM returns response:

Certainly! I'd be happy to help you with Java programming. Could you please provide more specific information about what you'd like to know or do with Java? Are you looking for general information about Java, a specific concept, or perhaps help with writing a particular Java program?

To get us started, here are a few common topics related to Java:

1. Basic syntax and structure
2. Object-oriented programming concepts in Java
3. Data types and variables
4. Control flow statements (if-else, loops, etc.)
5. Methods and functions
6. Classes and objects
7. Inheritance and polymorphism
8. Exception handling
9. Java Standard Library (java.util, java.io, etc.)
10. Java development environment setup

If you have a specific question or topic in mind, please let me know, and I'll be glad to provide more detailed information or code examples.

Do you have any particular area of Java you'd like to explore or any specific questions? Your feedback will help me provide a more targeted and helpful response.

but ReActPlainTextParser returns null

Possible fixes

Add post processing for whole LLM response. If answer doesn't have Final answer field, but doesn't match Action either, just return whole message as a final answer.

We have this functionality in Rails

Note, that in Rails in V1 we also don't stream answer if we don't have Final Answer: part. In this case, the answer will appear in the chat window only when streaming is over.