<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=16838) </details> <!--IssueSummary end--> Our current syntax highlighting setup is not efficient at all. For example, when we want to display a file or diff we simply load the entire thing into memory and highlight it. In case of diffs it's worse, we load/highlight both the old and new version. This can lead to large amounts of memory being consumed (as described in https://gitlab.com/gitlab-org/gitlab-ce/issues/26329). To make this more efficient we need to be able to stream input to our syntax highlighter, e.g. one line at a time. The highlighter in turn should spit out only a single chunk/line at a time. Basically you'd end up with an interface such as: ```ruby lexer = Rouge::LexerThing.new(input_io_object_that_acts_like_an_enumerable) while line = lexer.next_line # do something with "line" end ``` cc @jneen