IssueReader/EpicReader tool selection failure when current page is issue/epic
Currently IssueReader/EpicReader tool selection can fail sometimes.
When asked Can you list all the labels on this issue?
(Colab), the
success rate is 80%
The failure case does not explicitly mention the tool name, but does instead identify the <resource>
tag to work with, which kind of makes sense:
The question is asking to list all the labels on the current issue the user is viewing. I have access to issue data through the tags. I can use this information to directly answer the question.\n\nAction: DirectAnswer\n\nFinal Answer: The labels on the current issue are:\n- ai-enablement
Proposal
We could experiment with adding further instruction such as:
When this data is useful, indicate it by selecting "IssueReader" tool.
However I wonder if we need to include the issue data in the prompt at all? Later on, issue identifier should be able to include this data anyways.
Also the following discussion from !146880 (merged) should be addressed:
-
@lulalala started a discussion: (+1 comment) suggestion: Would this makes this more grammatically correct? (Same with the Epic)
The user is currently on a page that displays an issue with a description, comments, etc., which the user might refer to, for example, as 'current,' 'this,' or 'that'.