Add Syntax Testing Functionality
Ciao André,
I was thinking that it would be really nice to add to Highlight syntax testing functionality. The idea came to my mind by seein how Sublime Text implements a simple way to test custom syntaxes during developement:
This could be implement via a new command line option:
highlight --test=<test-files> [--langdef=<langDef-file>]
Where <test-files>
could be a single file, a files pattern, or a folder; and the --langdef
option serves when testing either a new syntax unknown to Highlight's configuration or a different version from the official one.
Pretty much along the lines of how Sublime Text implements testing syntax, the user will create test files following simple rules:
- Test files' filenames should start with
langdef-test_
- They'll keep their native file extensions (and Highlight will use it to determine which syntax they should be tested against)
- The first line in a test file is scanned by Highlight to gather some settings information:
- Encoding (if BOM is found)
<comment delimeter>
-
'syntax'
syntax to use (optional, to resolve conflicts when the file extension is shared by multiple syntaxes). -
"path-to-langDef"
to use (optional, useful for overriding default langDefs) - Other inline options (optional, as an alternative to command line switches)
So, the first line of a C++ test file (eg, "langdef-test_edge-cases.cpp
") should look something like:
// 'Cpp' "../updated_Cpp_syntax.lang"
As for the test-file contents, it's just a real sourcecode file with additional comment lines which contain symbols to delimit tokens in the source line above them, followed by the expected highlighting token.
I'm pasting this from Sublime Text documentation:
Each test in the syntax test file must first start the comment token (established on the first line, it doesn't actually have to be a comment according to the syntax), and then either a
^
or<-
token.The two types of tests are:
- Caret:
^
this will test the following selector against the scope on the most recent non-test line. It will test it at the same column the^
is in. Consecutive^
s will test each column against the selector.- Arrow:
<-
this will test the following selector against the scope on the most recent non-test line. It will test it at the same column as the comment character is in.
I would also add:
- The
//
can be used as a further comments delimiter (regardless of the delimiter used by the file), i.e. to allow further commenting tests. Example:
Debug "k = " + Str(k)
; ^^^^^^ string // FIXME: String not being highlihgted correctly!
With Highlight, the name of the CSS classes could be used for comparing the expected result with the true result. Example:
; 'PureBasic' "./purebasic_work.lang"
For k = 0 To 10
; <- kwa // The '<-' captures "For"
; ^ opt
; ^ ^^ num // NOTE: this line carries out two tests at once ("0" and "10")
Debug "k = " + Str(k)
; ^^^^^^ string // FIXME: String not being highlihgted correctly!
; ^ operator
Next
; <- kwa
Of course, being Sublime Text syntax different from Highlight (which doesn't nest scopes), the above ideas are just a proposal and a starting point. Possibly, the fact that Highlight doesn't nest elements could allow more flexibility in the comments system employed.
What I like of this approach is that you are dealing with real sourcecode files, which can be actually compiled; so you can ensure that syntax test files are also legitimate files in the language at hand.
The system is simple and it could be used to automate testing of multiple langDefs at once, uncovering overseen edge cases and, especially, being able to see the impact of small changes to syntax files (especially with nested syntaxes).
I understand that this new feature might be a rather big change, but it would be good if it was part of Highlight's binary, rather than implemented as a separate and independent tool. What do you think about it?