Allow extractor Singer messages to be transformed using Python before passing them to loader (Stream Maps)
We could support transformer
plugins other than dbt
, that would follow extraction rather than loading, and would transform an extractor's output stream of Singer messages (SCHEMA
, RECORD
, etc), before they're streamed into the loader.
The two types of transformers could be distinguished using an extra, e.g. type: {etl,elt}
or {follows,acts_on,transforms}: {extractor,loader}
We could support both pip packages, and local executable files, using either the pip_url
or executable
plugin setting.