Soften application limits for diffs
## Problem to solve
GitLab implements hard diff limits, which prevent large merge requests from being viewed. The hard limits are problematic in certain situations where a customer wants to merge changes from an upstream project (Linux kernel, networking stack etc) or reviewing changes to generated code or configs.
The work around is to view the raw diff.
## Further details
Right now, diff collapsing is kind of a mess:
1. We collapse individual diffs over 100 KB.
2. We collapse diffs after the diffs rendered so far have reached:
1. 100 files.
2. 5,000 lines.
3. 500 KB.
Changing this is pretty painful. However, with diffs in MRs having their own table, and diffs for commits and compare being moved to Gitaly, we can be smarter. For instance, we could handle the collapse limits in Gitaly, or in the database (although that would need some SQL trickery that might not work out any faster than doing this in Ruby.)
We could also do 'infinite scroll' for diffs - we could actually do this now, but it's easier if we aren't loading the whole thing in Ruby some of the time anyway - where we load all the filenames, and the 'first page' of content initially, then have the frontend handle the rest of the content.
## Proposal
Make the interface capable of progressively disclosing more information:
- infinite scroll
- file tree
## Customers
- https://gitlab.my.salesforce.com/0016100001Eo8P8AAJ
- https://gitlab.my.salesforce.com/0016100000W44Pc
- https://gitlab.my.salesforce.com/0016100000Nm6sS
- https://gitlab.my.salesforce.com/00161000004xhv1AAA
- https://gitlab.my.salesforce.com/00161000005cxCtAAI
- https://gitlab.my.salesforce.com/00161000002xBZGAA2
epic