Skip to content

Adding Transform endpoint

Alex Zamai requested to merge transform-endpoint into master

After trying different approaches and discussion with @iroussos I stelled on the following scenario for running dbt run and passing the database connection values to it:

  1. The dbt main function is invoked as part of the /transofrm/<model_name> API endpoint request handler
  2. The dbt main function is run from the transform folder which mimics the way it is run on the cli.
  3. The connection configuration is specified in the profiles.yml file which are referencing the env variables.
  4. Within the request, handler connection values are pulled from the settings table and env variables are updated by the runtime, to be later consumed by the dbt run execution.

I realize this is not the most elegant or future-proof solution, but I think that dbt transform would be mostly run on the scheduled job/interface which would use cli style configuration and credentials value passing, and the invocation from the UI(frontend) is secondary use case and can be limited in its capabilities.
@jschatz1 @iroussos

Edited by Melty Bot

Merge request reports