add warnings for unreasonable input sizes
Fixes: #631 (closed)
Adds warnings for parsing unreasonably large katscript (both in total size and maximum line size)
It is not easy to support arbitrarily large input lines, since the parses uses recursion by design and python's recursion limit is quite conservative.
Finesse supports infinitely sized katscript in principle, in practice you are rather limited by compute and memory limits: parsing 200kb of just mirror elements takes 40 seconds and 8 gb of memory on my machine. This might actually not be a limit of the parser, but the builder creating the python structures for all these mirrors.
Line length warnings
Input
import finesse
phases = 600
model = finesse.Model()
model.parse("m m1 R=0.9 T=0.1")
model.parse(f"sweep(m1.phi, {[i for i in range(phases)]}, True)")
Output
/home/miron/src/finesse3/src/finesse/script/parser.py:142: KatParsingInputSizeWarning: Unreasonable line length: 602 words in line 0.
Low performance or recursion error possible.
Content: 'sweep(m1.phi, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,...'
warnings.warn(msg, KatParsingInputSizeWarning, stacklevel=1)
Making to long of a line (setting phases
to 1200 on my machine)
/home/miron/src/finesse3/src/finesse/script/parser.py:142: KatParsingInputSizeWarning: Unreasonable line length: 1202 words in line 0.
Low performance or recursion error possible.
Content: 'sweep(m1.phi, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,...'
warnings.warn(msg, KatParsingInputSizeWarning, stacklevel=1)
Traceback (most recent call last):
File "/home/miron/src/finesse3/test.py", line 7, in <module>
model.parse(f"sweep(m1.phi, {[i for i in range(phases)]}, True)")
File "/home/miron/src/finesse3/src/finesse/model.py", line 78, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/miron/src/finesse3/src/finesse/model.py", line 2229, in parse
parse(text, model=self, spec=spec)
File "/home/miron/src/finesse3/src/finesse/script/__init__.py", line 44, in parse
return compiler.compile(text, model=model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/miron/src/finesse3/src/finesse/script/compiler.py", line 154, in compile
return self.compile_file(StringIO(string), **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/miron/src/finesse3/src/finesse/script/compiler.py", line 197, in compile_file
script = self._parser.parse_file(fobj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/miron/src/finesse3/src/finesse/script/parser.py", line 120, in parse_file
raise KatParsingError(msg)
finesse.script.exceptions.KatParsingError:
Recursion limit reached during parsing. Probably caused by an unreasonable line length.
Try increasing recursion limit with `sys.setrecursionlimit` or use the python interface.
Total file size
input
import finesse
n_lines = 1000
# 1000 lines is twice the size of the finesse ligo model (500 lines ~10kb)
model = finesse.Model()
model.parse("\n".join(f"m m{i} R=0.99 T=0.01" for i in range(n_lines)))
output
/home/miron/src/finesse3/src/finesse/script/parser.py:149: KatParsingInputSizeWarning: Unreasonable script size: 20 kb. Low performance or recursion error possible.
warnings.warn(msg, KatParsingInputSizeWarning, stacklevel=1)
Edited by Miron van der Kolk