Improve merge request summary prompt
Problem to Solve
We've received lots of feedback that the merge request summaries aren't quite right. They're close, but not all the way there.
Proposal
We should spend some time tweaking the prompt (potentially with the examples we have of MRs) to see if we can get closer to accurate. We could also evaluate other models, but starting with just the prompt might be enough.
Spike
We're going to time box this as a spike spending about 2-3 days just testing if there are improvements we can make to the prompt. If there are, great, if there aren't we'll come back again in the future.
Test Data
Due to the feedback issue we have quite a few example MRs, the summary that was given and what was wrong. We should try again with this same data to see if we can get closer to what the users expected. Some (non-exhaustive) examples: