Skip to content

Implement Inko's parser in Inko

Yorick Peterse requested to merge std-compiler-parser into master

Details

The parser is an LL(1) recursive descent parser. While the Ruby parser was used as a reference, it is not a 1:1 port.

As part of porting the parser to Inko, several syntax changes were made:

  1. "where" on methods is not supported by the Inko parser. When we start using the parser we will move "where" to traits.

  2. The comments "#!" and "##" are replaced with just "#". This makes the lexer and parser internals easier, and removes the need for having to remember three different comment types.

  3. Mutable rest arguments are supported in the Inko parser using "mut NAME", instead of " mut NAME".

Using the parser is straightforward:

import std::compiler::parser::Parser

let parser = Parser.new(input: '10 + 2', file: 'test.inko')

try! parser.parse

The AST nodes are located in modules under std::compiler::ast. For example, integer literals are in std::compiler::ast::literals. For now there is some minor duplication across some of the AST nodes. For example, the types of methods (MethodDefinition) and required methods (RequiredMethodDefinition) are similar. When we start implementing the compiler we may change this around a bit, or perhaps split the AST nodes into more separate types.

Fixes #172 (closed)

TODO

Tokens to support as a value:

  • attribute
  • colon_colon
  • constant
  • define
  • identifier
  • let
  • return
  • self
  • static
  • throw
  • try
  • try_bang
  • slicing syntax
  • comment
  • curly_open
  • do
  • documentation
  • float
  • integer
  • lambda
  • paren_open
  • string

Other tasks to complete:

  • Parse required methods
  • Parse comments inside parentheses
    • This brings up an interesting question: how should we handle comments? Module/documentation comments can only occur in certain places, but regular comments can go everywhere (e.g. inside parentheses). These comments should be available somehow so we can go from source to AST and back, without losing them.
  • Parse rest arguments
  • Parse mutable arguments
  • Parse import wildcards
  • Run the parser over the runtime to see if it parses everything.
    • ⚠ Since it does not support method bounds it will error out on code that uses this.
  • Organise the parser code a bit better so there are fewer AST modules and random helper methods

Corresponding issue

#172 (closed)

Checklist

Edited by Yorick Peterse

Merge request reports