Commit 78f15e8b authored by ArthurH's avatar ArthurH

added documentation for lexer and domain

fix role typo for aimms:handle
parent 51494ba8
......@@ -3,10 +3,10 @@
This Sphinx theme was designed to provide a great reader experience for documentation users on both desktop and mobile devices for AIMMS projects.
**This theme also includes:**
- an **AIMMS [pygment lexer](http://pygments.org/)** to highlight your AIMMS [code blocks in sphinx](http://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-block)
- an **AIMMS [Domain](http://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html)** to document your own AIMMS code.
- an **[AIMMS pygment lexer](docs/AIMMS Lexer.md)** to highlight your AIMMS [code blocks in sphinx](http://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-block)
- an **[AIMMS Domain](docs/AIMMS Domain.md)** to document your own AIMMS code.
Please find the documentation of those 2 elements in the Wiki.
Please find the documentation of those 2 elements following the links above.
This theme is inherited from the great [Read the Docs](https://github.com/readthedocs/sphinx_rtd_theme) but can work with any Sphinx project.
......
AIMMS Domain for Sphinx
=========================
As first introduction to domains, please refer to [Sphinx Domain Docs](http://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html) to document your own AIMMS code.
This AIMMS Domain is build up from the [Javascript Domain] (https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#the-javascript-domain)
Basic Usage
============
```
.. aimms:parameter:: spam(eggs)
Spam or ham the foo.
```
This describes an AIMMS parameter called spam indexed over the eggs index.
The domains also provide roles that link back to these object descriptions. For example, to link to the parameter described in the example above, you could say
The function :aimms:parameter:`spam` does a similar thing.
As you can see, both directive and role names contain the domain name and the directive name.
The AIMMS Domain
===================
The AIMMS domain (name aimms) provides the following directives:
.. aimms:module:: name
This directive sets the module name for object declarations that follow after. The module name is used in the global module index and in cross references. This directive does not create an object heading like py:class would, for example.
By default, this directive will create a linkable entity and will cause an entry in the global module index, unless the noindex option is specified. If this option is specified, the directive will only update the current module name.
.. aimms:function:: name(signature)
Describes an AIMMS function. If you want to describe arguments as optional use square brackets as documented for Python signatures.
You can use fields to give more details about arguments and their expected types, errors which may be thrown by the function, and the value being returned:
.. aimms:function:: MyFunction(href, callback[, errback])
:attribute string href: An URI to the location of the resource.
:attribute callback: Gets called with the object.
:attribute errback:
Gets called in case the request fails. And a lot of other
text so we need multiple lines.
:throws SomeError: For whatever reason in that case.
:returns: Something.
List of supported types, following the same rules as ``aimms:function::`` :
aimms:function::
aimms:procedure::
aimms:externalprocedure::
aimms:parameter::
aimms:elementparameter::
aimms:stringparameter::
aimms:set::
aimms:variable::
aimms:constraint::
aimms:mathematicalprogram::
aimms:file::
aimms:handle::
List of supported types, following the same rules as ``aimms:module::`` :
aimms:module::
aimms:librarymodule::
These roles are provided to refer to the described objects:
:aimms:func:
:aimms:procedure:
:aimms:librarymodule:
:aimms:externalprocedure:
:aimms:parameter:
:aimms:elementparameter:
:aimms:stringparameter:
:aimms:set:
:aimms:variable:
:aimms:constraint:
:aimms:mathematicalprogram:
:aimms:file:
:aimms:handle:
\ No newline at end of file
AIMMS Lexer - Syntax highlighter
------------------------------------
an **AIMMS [pygment lexer](http://pygments.org/)** to highlight your AIMMS [code blocks in sphinx](http://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-block)
Intro
---------
The AIMMS lexer is meant to highlight any aimms code block in our RST files, according to AIMMS procedure script IDE. It supports AMS files as well, i.e. **Text representation** of an AIMMS model.
To use is it, you should use the `code-block` directive, as follows:
```
.. code-block:: aimms
P_Demand(i) := Uniform(1,200)
```
See an example [here](https://how-to.aimms.com/Generate%20Random%20Numbers.html#csvinterface-ams)
Currently supported prefixes and their colors (**case insensitive**)
-------------------------------------------------------------------
| Prefix | Type | Coloring |
|----------|----------------------|----------------|
| P_ | Parameter | DarkCyan |
| P01_ | Parameter | DarkCyan |
| EP_ | ElementParameter | DarkCyan |
| S_ | Set | black |
| Cal_ | Set | black |
| SP_ | StringParameter | black |
| V_ | Variable | PaleVioletRed |
| V01_ | Variable (binary) | PaleVioletRed |
| EV_ | ElementVariable | PaleVioletRed |
| C_ | Constraint | black |
| MP_ | MathematicalProgram | black |
| DBT_ | DatabaseTable | black |
| PR_ | Procedure | black |
| CNV_ | Convention | black |
| I_ | Index | black |
Current supported Types and their associated color (based on AIMMS color IDE)
-----------------------------------------------------------------------------
| Priority | Type | Coloring |
|----------|-----------------------------|---------------------|
| 1 | Comment in procedure | green |
| 2 | Comment in Attribute Window | green |
| 3 | AllKeywords (if,then..) | blue |
| 4 | AllPredeclaredIdentifiers | brown |
| 5 | String ("") | gray |
| 6 | Element ('') | gray |
| 7 | AllIntrinsicFunctions | brown |
| 8 | Identifier prefix | various (see above) |
How-To customize Highlighted colors
--------------------------------------
For now on, every colors are defined in the [Hacks.css file](https://gitlab.aimms.com/aimms/documentation/blob/master/_static/Hacks.css). You may scroll down till the end of the file, and see the basic css there.
Please be aware that modifying the CSS file is not considered as a change in any RST file, thus **Sphinx will not rebuild the
doc**. In order to force sphinx to append the new css file, just make a dummy change to the file you want to rebuild (add a space, remove it and save the rst document, that's sufficient).
Current types that are "supported"
-----------------------------------
If you look at the bottom of the Hacks.css file, you should see something like:
``` css
/*-------- Styling for AIMMS declared identifier name by type (see AIMMS PowerPoint colour palette) --------------*/
/*Set*/
div.highlight-aimms span.n.n-Set {color:black;}
/*Parameter*/
div.highlight-aimms span.n.n-Parameter {color:DarkCyan;}
/*Element Parameter*/
div.highlight-aimms span.n.n-ElementParameter {color:DarkCyan;}
/*Variable*/
div.highlight-aimms span.n.n-Variable {color:PaleVioletRed;}
/*Constraint*/
div.highlight-aimms span.n.n-Constraint {color:black;}
/*MathematicalProgram*/
div.highlight-aimms span.n.n-MathematicalProgram {color:black;}
/*String Parameter*/
div.highlight-aimms span.n.n-StringParameter {color:black; }
/*Quantity*/
div.highlight-aimms span.n.n-Quantity {color:black;}
/*DatabaseTable*/
div.highlight-aimms span.n.n-DatabaseTable {color:purple;}
/*Index*/
div.highlight-aimms span.n.n-Index {color:black;}
/*Convention*/
div.highlight-aimms span.n.n-Convention {color:black;}
```
As you can see, this CSS is referring to some particular class (css selector), for example ``n.n-Parameter``. Since it is CSS styling, you can check your file directly in the browser to see which selector are used (as in AIMMS WebUI):
![image](/uploads/68c9b08a6beb1677bbe407f75cd562a1/image.png)
Current types that are recognized and passed to the HTML are the following:
* Parameter as `n.n-Parameter`
* ElementParameter `n.n-ElementParameter `
* Set `n.n-Set`
* StringParameter `n.n-StringParameter`
* Variable `n.n-Variable`
* Constraint `n.n-Constraint`
* MathematicalProgram `n.n-MathematicalProgram `
* DatabaseTable `n.n-DatabaseTable`
Default rules used to highlight text
-------------------------------------
Everything happens in the file ``aimms.py``, located in `_static/AIMMSLexer/Lexer/aimms.py`. I will describe it from top to bottom.
This syntax highlighter is using regular expressions. In this file, we have several regular expression rules, one after another that will detect words, and return the corresponding type (used to show a style,color,font).
**As soon as a regular expression is valid (that there is a match), the procedure breaks (exit) and returns the found match type.** This logic enables us to describe the process as follows:
If the text is a comment single line, multiline, inside a procedure or in a comment block of any attribute window:
> The text is green.
if not, Sphinx checks if it is an AIMMS keyword (blue type, like `if` or `then` )
> the text is blue
etc. ...
The priority list is thus as above:
| Priority | Type | Coloring |
|----------|-----------------------------|---------------------|
| 1 | Comment in procedure | green |
| 2 | Comment in Attribute Window | green |
| 3 | AllKeywords (if,then..) | blue |
| 4 | AllPredeclaredIdentifiers | red |
| 5 | String ("") | gray |
| 6 | Element ('') | gray |
| 7 | AllIntrinsicFunctions | red |
| 8 | Identifier prefix | various (see above) |
Regular expression resources to learn and use them
------------------------------------------------------
More info about regular expression follows:
- https://docs.python.org/2/howto/regex.html#regex-howto
- http://pygments.org/docs/lexerdevelopment/?highlight=words
A wonderful website to check your regular expression is https://regex101.com/r/dU5fO8/33
You may change any rules in this `aimms.py`. Mind the comma at the end of your rule :smile:
Existence check for declared identifiers: a hack in ``get_token_unprocessed``
------------------------------------------------------------------------------
After the priority list has been run through, the Lexer will **re-scan the entire code block** a last time to isolate any identifier names declaration, and highlights other identifier names occurrences in the entire code block with regards to their type (Parameter is DarkCyan, StringParameter is black, Variable is PaleVioletRed etc.).
The code for this last behavior is in the ``get_tokens_unprocessed`` function (at the end of the `aimms.py`)
> this makes sense, since names are unique in AIMMS
How To improve or correct the lexer (`aimms.py`) ? The $ symbol use case
--------------------------------------------------------------------------
Recently, we saw an obvious bug in the AIMMS lexer: the $ symbol, also called Sparsity Modifier, was not recognized by the lexer. This has resulted in a red box around every $ symbols part of an AIMMS code block.
![image](uploads/056fbaf5b265b6ef9d332ff0b775cf71/image.png)
(remark: when an expression is not matched with any regular expression shown previously in [Default rules used to highlight text](https://gitlab.aimms.com/aimms/documentation/wikis/advanced-use/aimms-lexer-for-syntax-highlighting#default-rules-used-to-highlight-text), this red box behavior will occur)
To fix it, we had to modify a regular expression rule from the lexer, contained in `aimms.py`.
Our rule of interest was the **AIMMS mathematical operators** rule, matching and highlighting any AIMMS operators (`:=,=,<,>,(,),{,},`...), bold and black.
``` python
#Highlights mathematical operators
(r'(\+|\-|\*|/|\*\*|=|<=|>=|==|\||\^|<|>|\!|\.\.|:=|\&|\!=|<<|>>|;)', Operator),
```
We finally just had to make the $ symbol part of this list, by using the "or" logical regex operator `|`. We also had to mind the escape sign `\` in front of `$`, because the $ has also a custom role in regular expressions (regex).
``` python
#Highlights mathematical operators
(r'(\+|\-|\*|/|\*\*|=|<=|>=|==|\||\^|<|>|\!|\.\.|:=|\&|\!=|<<|>>|;|\$)', Operator),
```
Et voilà !
\ No newline at end of file
......@@ -475,7 +475,7 @@ class AIMMSDomain(Domain):
'constraint': AIMMSXRefRole(fix_parens=True),
'mathematicalprogram': AIMMSXRefRole(fix_parens=True),
'file': AIMMSXRefRole(fix_parens=True),
'hamdle': AIMMSXRefRole(fix_parens=True),
'handle': AIMMSXRefRole(fix_parens=True),
'data': AIMMSXRefRole(),
'attr': AIMMSXRefRole(),
'mod': AIMMSXRefRole(),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment