DCR has been returning "...nothing to comment on" for a lot of MRs
Problem
We're noticing that DCR has been returning ...nothing to comment on
for a lot of MRs following the recent prompt changes. So far, we haven't found any logs where requests are failing or anything like that. To be sure, we need to double check nothing is broken for DCR.
Just a hunch but maybe DCR is setting priority to less than 3
for the comments, given that it has a lot more context because of MR title, description and access to all diffs in one go? Do we store the actual response from the LLM somewhere? If we can see those logs then we can check all comments by DCR and how many of them we assign priority less than 3
and got filtered out.
Feedback link: #517386 (comment 2476817635)
Proposal
Investigate, ideate solutions, and implement them.
- Post medium priority comments if there are no high priority comments
- Improve summary comment when there are no comments, by including the why: #541694 (closed)
- Add logging to see the distribution of P1/2/3 comments !191272 (merged)
- ...
Edited by Sincheol (David) Kim