Skip to content

JSONPath expressions don't support extensions

Summary

JSONPath expressions for records and pagination tokens (see source) don't support JSONPath extensions such as filters, sorting, etc.

Steps to reproduce

Try using an expression with a filter, such as $.link[?(@.relation=='next')].url and the tap execution will fail.

What is the current bug behavior?

JSONPath extensions are not supported.

What is the expected correct behavior?

JSONPath extensions are a useful feature and should be supported.

Relevant logs and/or screenshots

NA

Possible fixes

According to the jsonpath-ng docs, extensions are supported using the extended parser:

>>> from jsonpath_ng.ext import parse

# A robust parser, not just a regex. (Makes powerful extensions possible; see below)
>>> jsonpath_expr = parse('foo[*].baz')

As a first step, a test case should be added here: https://gitlab.com/meltano/sdk/-/blob/241b38e8b6cdfa6f3397d9c70e7b9082b4594554/tests/core/test_streams.py#L156-225

Edited by Edgar R. Mondragón