Support SaaS targets for "Reverse ETL/ELT" and "Operational Analytics" use cases
A common use case for data teams is to move data from their data warehouse, after it's been transformed, and into a downstream SaaS application. This as come to be known as Reverse ETL or Operational Analytics. Two of the newer startups in this space are Census and Hightouch.
Meltano and Singer can be a viable solution to this space. Each SaaS target would have to handle how to transform data and it's a use case that's been thought about for the SDK.
On the Meltano side, we'd want to support this with a good definition experience in meltano.yml. A first iteration already exists in that this would be an extractor
and loader
, but since reverse etl typically occurs after transformation, it may be useful to have an alias on these (export / publish?). Users may want to import credentials from an existing target, which could imply some kind of chaining of an extractor to a loader. We'd have to be more aware of the DAG then.
Meltano could also be made aware of dbt exposures and understand the DAG between transformation and another extract.