Encourage users to have smaller merge requests
Problem to solve
Microsoft found that reviewers with experience reviewing or editing the file provided more useful feedback.
Code review usefulness is negatively correlated with the size of a code review. That is, the more files there are in a single review, the lower the overall rate of useful feedback. The decrease however only starts to be noticeable for reviews with 20 or more changed files.
Low quality feedback reduces the value of performing a code review.
Intended users
Further details
From gitlab-design#441 (closed).
How can we encourage the best practice of small changes that are easier to review and merge? One possibility is showing that big merge requests take X amount of time on average to review.
Large, complex changes are among the most popular reasons for confusion […]. As a strategy to deal with this issue, developers suggest to split big changes into smaller, clearer ones. This is consistent with the earlier findings, envisioning the emergence of tools enabling early detection of splittable changes, i.e., before the pull request is submitted, in order to both avoid spending additional time in identifying such patches and asking the author re-work the change to reduce its complexity and likelihood of the discussion. — Confusion in Code Reviews: Reasons, Impacts, and Coping Strategies
Another point raised during the interviews is that large patch sets are difficult to review and require a lot of time to read [P1−4, FG1−3], thus this may delay the acceptance of the patches. […] Interestingly, all the interviewees agreed that tools could help reviewers and authors in solving this need: for example, when submitting a large patch, the tool could suggest the author to split it into more parts to ease the reviewing process. — Information Needs in Contemporary Code Review
Observation 4 shows that as we conjecture, churn shares an increasing relationship with the likelihood that a patch will be discussed. Recent studies also report that the patch size is a good indicator of patch acceptance (Weißgerberet al., 2008; Jianget al.,2013). Intuitively, the large patches are likely to be discussed since they can contain more problems than the small patches. On the other hand, smaller patches have less code to critique, and thus are less likely to receive comments from reviewers. This observation is also consistent with the findings of Baysalet al.(2015) who find that large patches tend to have more revisions than small patches […]. — Review Participation in Modern Code Review
Proposal
Show a warning when a merge request changes more than 20 files. This should:
- warn the author and encourage them to split the change into smaller merge requests
- warn any reviewer that their review is less likely to catch all errors
The warning should create social pressure against large changes which reduce velocity and quality.