Skip to content

Grammar tweaks

finesse importer requested to merge feature/grammar-tweaks into master

Fixes #89 (closed), #104 (closed), #105 (closed), #109 (closed), #114 (closed), #115 (closed), #116 (closed).

Related to #113 (closed).

This MR overhauls the parser to support the suggestions made in #105 (closed). To summarise:

  • Expression/parameter list disambiguation (see #105 (closed)) is achieved by insisting that unary expressions (e.g. the -5 part of 5 + -5) have no whitespace.
  • Strings now need to be in quotes, e.g. "my string" or 'my string', unless a reserved keyword like lin, log, am, pm, etc.
  • The "kat script spec" now defines the available "namespaces" like "model", "laser", "beamsplitter", "analysis", etc. These in almost all cases map directly to either a class (e.g. Laser) or property (e.g. Model.lambda0) and this structure makes it straightforward for the parser and generator to build models and scripts. If not specified, the namespace defaults to "model", so commands like lambda 1064e-9 still work (which would otherwise be model lambda 1064e-9).
  • Analyses can now be defined starting with "analysis", e.g. analysis xaxis &l1.P lin 1 5 100. This is because of the "namespace" change above. They can however still be directly named (e.g. xaxis) as long as xaxis is unambiguous, i.e. not also a field of another namespace.
  • Tag parsing support was removed (they were anyway thrown away by the parser). We should think if/how these should be supported.
  • Adds support for boolean keywords true and false (no other forms are accepted). This is useful for kat script generation, where bool("false") == True in Python, which can be misleading.
  • Import in finesse.__init__ removed so that the parser grammar doesn't get built every time Finesse is run.
  • Added an analysis property to Model so it has a proper docstring for the adapter class.
  • Custom elements (e.g. components and detectors), analyses and functions can now be registered with KatSpec.register_element, etc. This automatically adds parser support for the session.
  • Added loads of new syntax tests. These tests also demonstrate the use of mock objects with Pytest, in this case to monkey-patch the parser to avoid building the parsed instructions (so that syntax like arrays can be checked before they get built into the model).

The "namespace" addition will also allow us to easily add support for other commands that don't map to equivalent Model functions, like e.g. logging, plotting, etc. as discussed in #105 (closed). If others don't like this change, we could get rid of it, but I think it's a logical way to organise things.

The import code was removed because we decided it needs more thought. I updated #100 (closed) to include the information we discussed. This import feature should not block this merge request, but rather be something we work on in the future.

Some other notes:

  • Strings can currently contain newline characters. It might need a more complicated regular expression. I also don't think entering a new state to tokenise strings is really needed, we can probably do it with a single regex. (fixed)
  • Some of the parser error messages need improvement. I have worked on this on the !19 (merged) branch so these will eventually get merged in too.
  • There are now 36 parser shift/reduce conflicts, up from 8. These are not in theory a problem, but we might want to refactor the grammar rules later to get that number down. We can also specify precedence rules as per the SLY documentation.

To-do from Thursday call and other discussions:

  • Keep the import stuff out of the parser for now. If we implement it in the future, it should be made to work first in Python and then later we'll add parser support. See #100 (closed).
  • Make KatSpec a singleton and use instance fields to store the supported components, functions, etc., so that users can add support for custom components and functions.
  • Allow custom components and functions to be defined in KatSpec so the parser can handle them.
  • Add tests testing the registration of custom components and fields with the parser.
  • Support multi-parameter functions, comma delimited
  • Allow pi as a symbol
  • Don't eagerly evaluate cos(2), etc.
  • Infer the namespace from the field if not specified. Check all available namespaces for the specified field, and if it doesn't resolve to exactly one namespace then throw an error. This allows e.g. "xaxis" to work on its own without "analysis".
  • Fix #115 (closed).
  • Add floor division and power operators (#116 (closed)).
  • Either write user docs showing how to add custom components, or make an issue as a to-do (milestone: beta 1)
  • Make state machine diagram for parser (hard)
  • Add tests for everything in #113 (closed)
Edited by finesse importer

Merge request reports