Change response format for Claude 3 to JSON
Note: This issue should be completed first #457211
Problem to solve
Right now, we get the response back from Claude 3 in the format of a string that is parsed.
This process is prone to error, and has caused bugs such as this.
Anthropic supports JSON responses, so we should try to switch over to that.
We should change the prompt to request a JSON response, and also extract the information from this JSON response rather than using the ChainOfThoughtParser
.
Note: there are multiple places that will need to be adjusted to change the prompt. See here
Note that this does not need to be supported for Claude 2.1, and should likely be behind a feature flag to protect against any regression.
Edited by Lesley Razzaghian