Pseudo Lang Servers

Ciao André,

I was thinking that the new LSP support in Highlight 4 opens the door also to a new possibility, i.e. implement "pseudo Lang Servers" for the sake of highlighting only (i.e. as an alternative to the native langDefs).

What I mean by this is that while implementing a full fledge LSP-compliant Language Server can be a quite complicate task — because it needs to be error tolerant and ensure real-time performance without slowing the editor — an ad hoc Lang Server created for highlighting source files with Highlight doesn't need to abide to such constraints, and might therefore be much easier to implement.

We can safely assumed that source files fed to Highlight are usually well formed, so pseudo Lang Server wouldn't have to deal with incomplete language constructs. Also, speed isn't critical in this use case, for the only goal would be to produce an highlighted document (nothing is being blocked in the process).

So, the LSP feature could indeed be exploited to cover those languages and syntaxes which are hard (or impossible) to implement via the native langDefs (e.g. because they need a stack to track semantic contexts).

It would also be great if Highlight 4 could provide a dedicate Lua API for implementing pseudo Lang Servers, allowing to create Lua Lang Servers that could be included in the Highlight distribution package, since they are not OS dependent (as opposed to binary Lang Servers).

I'm not sure if and this could be done via a Lua API, but if it could then it would ideally abstract away all the nitty gritty of having to deal with the JSON-RPC exchanges, allowing syntax authors to focus more (only?) on parsing the syntax. Since this API would be an easy alternative to implementing a Lang Server from scratch (which is always an option), it might even provide a fixed model, e.g. emulating TexMate syntaxes and how editors like Sublime Text and VSCode implement their syntaxes natively.

Maybe the new LSP feature could even be exploited to support TextMate grammars natively? i.e. if Highlight could parse a TextMate grammar via a dedicated Lang Server module which is able to transform them into a Stack based general purpose parser.

I would like to ear what you thing of these ideas. But I'm convinced that the basic idea of "pseudo Lang Servers" has its merits, and that they are much easier to implement than a full-fledged Lang Server intended for editors and IDEs.