Add fallback option for overflowed MRDs
What does this MR do?
When a MergeRequestDiff is "overflowed" (contains more than a certain
number of files, 1,000 as of %12.10) #modified_paths
will cut-off its
results at the last file of the limit (IE the 1,000th file.) For the
majority of MRs, this is fine, and we often have fallback code to use
Repository#diff_stats
to get at the expanded list (for example, in
lib/gitlab/code_owners.rb) This is less than ideal, as
Repository#diff_stats is slow.. like, super slow. Like, you shouldn't
use this unless you need to.
Since it is so slow, we really should have a single, unified implementation of this fallback. The consumer shouldn't have to special case their way around this limitation, and having it in 2 different locations (as of this comment) isn't great. We really shouldn't have multiple places calling their own implementations of code that is potentially very slow. By adding this fallback option, flagged to NOT run, we can then follow-on and remove our disparate and potentially divergent implementations, and provide a better option for future use.
Once this is merged, I'll followup with some tracking metrics, as well as an MR to drop the current fallback use cases in favor of this solution.
Related to #214733 (closed)