JSONPath expressions don't support extensions
Summary
JSONPath expressions for records and pagination tokens (see source) don't support JSONPath extensions such as filters, sorting, etc.
Steps to reproduce
Try using an expression with a filter, such as $.link[?(@.relation=='next')].url
and the tap execution will fail.
What is the current bug behavior?
JSONPath extensions are not supported.
What is the expected correct behavior?
JSONPath extensions are a useful feature and should be supported.
Relevant logs and/or screenshots
NA
Possible fixes
According to the jsonpath-ng
docs, extensions are supported using the extended parser:
>>> from jsonpath_ng.ext import parse
# A robust parser, not just a regex. (Makes powerful extensions possible; see below)
>>> jsonpath_expr = parse('foo[*].baz')
As a first step, a test case should be added here: https://gitlab.com/meltano/sdk/-/blob/241b38e8b6cdfa6f3397d9c70e7b9082b4594554/tests/core/test_streams.py#L156-225
Edited by Edgar R. Mondragón