Port fields parser and transformer module from Javascript to Rust

Problem

Preliminary reading:

Currently, the GLQL parser only parses the query1, i.e a vector of expressions that are a pair of field (key) / value pairs. The fields are included as a part of the context2 as a vector of strings. When called from Javascript using WASM bindings, we just split the fields on a comma.

In Javascript, a GLQL query is a block of YAML3. The yaml is parsed using JSYAML. The parsing of the query is delegated to the Rust parser, while Javascript code handles the parsing of the fields parameter. The fields parameter allows you to support transformation functions on a field. Currently only one field labels is supported4.

The fields parser in Javascript used to tokenize and parse a list of fields, with optional function transformations applied. Those transformations are applied on the response of the GraphQL API to derive new fields from existing fields in the GLQL table.

Implementation guide

  • Migrate the fields parser from javascript to Rust. That parser should convert a list of fields, with optional function transformations to an AST, similar to the query parser in Rust. The code should adopt a Rust style though.
  • Migrate the ast transformer from javascript to rust. This transformer transforms the AST of the fields parser by evaluating a function to its field name.
  • Use a YAML parser in Rust5 to parse GLQL config as YAML. Extract fields and query and pass them on to the fields parser and query parser respectively. Similar to the config parser in Javascript.
  • Migrate the data transformers in data.js, functions.js and field_aliases.js, that are each responsible for transforming data returned by a GraphQL response by its data source type, field transformation functions and field aliases respectively.
  • We expose a single function called compile_glql_wasm to compile the query itself. But the fields parameter is expected to be a comma separated list of fields. Keeping the external API unchanged, allow taking optional field function transformations, that the fields parser will take care of.

  1. Query syntax: A query looks like field [operator] value. Examples: assignee = "himkp" and label in ("label1", "label2"). Only AND is supported as a logical operator currently. ↩

  2. Query context: Query context may include, among other things, project, group, limit, offset (using before/after cursors in GraphQL) that can be useful to filter down the results. ↩

  3. Query in YAML frontmatter: Technically, the entire query block is not YAML. It is a mix of frontmatter and the query itself. See syntax section of the docs for more details. ↩

  4. labels field function: Used to select a single label from all labels on an issue. Can take infinite parameters. Examples: labels("workflow::*") and label('bug', 'feature'). ↩

  5. Parsing YAML in Rust: It seems that the top two options yaml-rust and serde-yaml are no longer maintained. The best option seems to be yaml-rust2. saphyr is more bleeding edge. ↩

Edited by Himanshu Kapoor