Generate syntax highlighted diffs file-by-file
Problem to solve
Syntax highlighting the merge request diff is slow because the diff for every changed file is highlighted all at once, and then the diff for every changed file is sent to the merge request interface in a single request. This makes the page slow to load.
We cache the data for the latest version of a merge request, and perhaps do some of this work when on the Git push, rather than waiting for page load (to be confirmed), but this also impacts the other comparisons like comparing arbitrary commits and versions which will not be cached in most instances.
Further details
This was a finding of the discovery sprint gitlab-foss#62536 (closed)
Proposal
When the new diffs API is implemented with support for pagination #31290 (closed), we should:
- refactor diff generation so that it can be generated for an individual file, multiple files, or all files
This will mean requesting a page of diffs only requires the diffs for those files to be syntax highlighted, not every single file in the diff.
Measuring success
TTFB should be less than 500ms for the diff endpoint when the cache is not primed (allowing 300ms to highlight one page of diffs)
This change is being made in coordination with &1816 which will improve performance of the merge request interface, and provide the foundation for reducing the frequency with which diff limits are hit https://gitlab.com/gitlab-org/gitlab-ce/issues/66638.