Draft: Trim tokens, expand patterns & modify arrangeParams

Adrien LESÉNÉCHALrequested to merge
Derugon/ParserPower2:master into master

WiP…

This merge request contains expansion changes requests and some small fixes. All changes have been tested on an empty test wiki (using MediaWiki 1.35), without additional Gamepedia extensions.

These changes are expected to break existing templates, so I don't really know if it would be possible to merge it or if it isn't possible at all.

Tokens trimming

Previous merge requests were made to allow additional spacing before and after parameters, and between list elements. For example, the list a, b would be parsed as [a, b] (using , as input separator), not as [ a, b].

However, if multiple tokens are given (on functions allowing the use of multiple tokens), spaces between tokens are not removed. For example, the following wikitext merges list elements having the same key (as <key1>:<value1>, <key2>:<value2>, …), concatenating the two values:

{{#listmerge:
 | list         = a:1, b:1, a:2, c:1, b:2, a:3, a:4
 | token1       = @fst,@snd
 | token2       = $fst,$snd
 | fieldsep     = :
 | matchpattern = <esc>{{#ifeq: @fst | $fst | yes }}</esc>
 | mergepattern = @fst:@snd$snd
}}

This wikitext is then parsed as a:1234, b:12, c:1.

But if we add additional spaces between the sub-tokens, we no longer get the expected result. For example, using @fst, @snd as token1 would give the @fst and @snd sub-tokens, and mergepattern would not recognize the @snd parameter. The result of the parser function would be a:@snd4, b:@snd2, c:1.

The tokens are splitted in the mergeListByPattern function (ParserPowerLists.php l.2386-2387):

				$tokens1 = explode($tokenSep, $token1);
				$tokens2 = explode($tokenSep, $token2);

Since array_map is used in other functions, it could be used to trim all sub-tokens:

				$tokens1 = array_map('trim', explode($tokenSep, $token1));
				$tokens2 = array_map('trim', explode($tokenSep, $token2));

Pattern expansion

The thing is: if we use |list={{a}} on #listmap or pretty much any list parser function, the {{a}} template is parsed, and if it returns 1, 2, 3, the list will be 1, 2, 3. But if we have a template b containing <div>@</div> and we use {{#listmap: |list={{a}} |token=@ |pattern={{b}} }}, we will get <div>@</div>, <div>@</div>, <div>@</div>.

The help wiki says:

For each item in the list, the token in pattern is replaced with the value and the result is returned. So for a token of @@@@ and a pattern of [[@@@@]], list values of Main page and Project:Community portal become [[Main page]] and [[Project:Community portal]].

After looking at the code, the pattern parameter is modified as follow:

  1. expand wikitext (do not expand templates and params.),
  2. replace the token(s) with the list element,
  3. reparse and expand the wikitext (with templates and params.),
  4. trim then unescape the text.

For example, if we use {{#lstmap: McMillen, Rodriguez | , | @ | User:@ }}, we get User:McMillen, User:Rodriguez. In this case, User: is added before any element of the initial list, as expected. We create a template 1 containing the parser function {{#lstmap: McMillen, Rodriguez | , | @ | {{{1}}} }}, and expecting the pattern as parameter. If we then use {{1 | User:@ }}, @ is not replaced because the token replacement in {{{1}}} is done before the expansion of the parameter.

To allow this, we could just change the first step above to also expand templates and parameters. If we have 2 templates fst and fnd containing respectively first and second, we can use {{#lstmap: fst, snd | , | @ | <esc>{{@}}</esc> }} and get first, second.

Merge request reports