WIP: Experiment/prototype approaches to caching ChangeAccess
What does this MR do?
Includes WIP code and comments I was experimenting with to make progress with https://gitlab.com/gitlab-org/gitlab-ce/issues/44679
Other notes
- Profile using Grafana and linux repo locally. For Grafana it helps to go to https://dashboards.gitlab.net/d/000000126/grape-endpoints?panelId=17&fullscreen&orgId=1&var-action=Grape%23POST%20%2Fapi%2Finternal%2Fallowed&var-database=influxdb-01-inf-gprd&from=now-2d&to=now and then narrow down on a timing spike. Not as useful as the old dashboard still.
Problem: we fetch raw commits in batches of 10 via
rev-list #{new_rev} --not --all
but then want to fetch diffs also in batches for a subset of those that by ignoring those which have cached resultsAlgorithm_v2: split by needs_diff? -> Process all with diffs -> Process all without diffs
Algorithm_v3: Get diffs in parallel in batches
Edited by James Edwards-Jones