Upload dbt artifacts on run end
Problem to solve
Meltano manages persisting dbt artifacts to some data store so users can easily leverage that metadata.
Target audience
Meltano users who also use the dbt integration to transform raw data loaded to a data warehouse target.
Further details
dbt
generates a number of artifacts over the course of a run
that are documented here. These artifacts contain useful metadata about the run itself, tables and views created by your dbt project your data warehouse, etc and can be used to power a number of use cases like analyzing run times and bottlenecks - see an example here.
Proposal
Idk
What does success look like, and how can we measure that?
One possible set of acceptance criteria would be that a Meltano user is able to specify a Cloud Storage bucket URI in meltano.yml and Meltano manages uploading any generated artifacts to the bucket. It might also be useful to be able to select what artifacts to upload.
Regression test
(Ensure the feature doesn't cause any regressions)
-
Write adequate test cases and submit test results -
Test results should be reviewed by a person from the team