Implement Inko's lexer in Inko
The Ruby compiler's lexer (also written in Ruby) should be ported to Inko, in the module
Inko's compiler will be gradually ported over from the current Ruby compiler to an Inko based compiler. The first step in this process is to port the lexer, followed by porting over the parser. The lexer will be available as part of the standard library, allowing future tooling to make use of it. For the foreseeable future the API will remain unstable, and as such might change at any point in time. As we get closes to 1.0.0 we will decide what will be part of the public API.
For the most part we'd just port the Ruby lexer to Inko, adjusting it to better fit Inko as a language where necessary.
The Inko lexer will likely be slower than the Ruby lexer, due to the complete lack of optimisations performed by Inko's compiler at the moment. This however is an opportunity for measuring Inko's performance and improving it in the future.