Introduce diffs_batch JSON endpoint for paginated diffs

What does this MR do?

Introduces the new diffs_batch.json endpoint without changing the existing diffs.json. This endpoint accepts the following arguments:

• page (default 1 if not given)
• per_page (default 20 if not given)

Benchmarks (before / after - 3rd and beyond are paginated):

• Gitaly calls are currently under 3
• Queries under 13
• Both below 50ms
• Most time is currently spent serializing the diff files

Remarks:

• We're not using the highlighted diff cache given how inefficient it is to write/read all cache at once. We'll be able to improve this further with #30550 (closed).
• We need to calculate how much we've loaded for diffs in previous files. That's necessary in order to know whether we're hitting the collection limits or not (commented at the code).

TODO

• Refactor
• Double check timings on Redis, DB and Gitaly (can we improve it further?)
• Add tests
• Add benchmarks
• Add profile data

Screenshots

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation created/updated or follow-up review issue created
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Performance and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers

#31290 (closed)