SPIKE: understand if switching tools to Anthropic function calling brings benefits in terms of answer quality and development efficiency
Anthropic recently released Tool use (function calling) in beta.
Tasks in this spike:
- We should investigate if this brings benefits to Duo Chat, such as
- improved success rate of picking the right tool (choosing the tool)
- improved success rate of running that tool (given correct inputs; and less error prone parsing)
- improved answer quality resulting from the two above
- development efficiency due to less effort with parsing
- other benefits
- We should investigate the effort of switching from our existing parsing solution to function calling?
- If the above is promising we should create (an) issue(s) for implementing the switch to function calling and bring it to workflowready for development
Development Notes:
- Please cap this effort to 2 or 3 days of research, provide a summary on the questions above and check-in with the EM/PM before investing further effort.
Edited by Juan Silva