Allow the highlight cache to be located in object storage
Problem to solve
GitLab displays highlighted code listings in a number of places - most prominently on the merge request diffs view, and when viewing an individual blob or commit.
The highlighting operation isn't cheap, and we store the results in the redis cache so we don't have to repeatedly calculate it. However, the sheer volume of data is challenging - and very expensive - to store in Redis.
Intended users
Instance administrators
Further details
The redis working set needs to fit in RAM, which makes redis our most expensive data store - even more expensive than the database. We shouldn't be routinely using it for storage of extremely large datasets.
In https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24276, we moved the storage of (unhighlighted) MR diffs from the database to object storage.
Proposal
Analogously to MR diffs, allow highlighted copies of blobs, MR diff files, and diffs for notes, to be placed in object storage instead of in Redis. This will be less performant, but - on large instances like GitLab.com - much more scalable.
cc @andrewn in case we want to make this one an gitlab-ce~9594407 request.
I don't know what percentage of the total redis cache set the highlight cache is, but I suspect it's significant. Quantifying it might permit us to prioritize this more accurately.