Skip to content

Add dbt clean before compile and runs

Emilie Schario requested to merge add-dbt-clean-before-running into master

Problem

Because of the way dbt works, every time you call dbt run it compiles the whole project. I noticed that if you had projects in your environment that were hanging around they would slow things down.

Solution

This MR cleans your dbt project before you run dbt deps. This is an extra command on every run, but it reduces the risk of compiling + building the DAG of a lot of queries that may not be relevant. I think it's one of those go-slow-to-go-fast things.

The dbt command is just dbt clean with no flags or specifications. Docs: https://docs.getdbt.com/docs/clean

Merge Request Checklist

  • Link to an issue #1551 (closed)
  • Include the proposed fix or feature
  • Include and update tests for the modified code
  • Include a documentation change
  • Add a CHANGELOG.md entry in the Unreleased section

Once your merge request is complete, please assign it a Meltano maintainer for review cycle. Once the review cycle finished, the reviewer shall approve the merge request so it can be merged by any Meltano maintainer.

Edited by Emilie Schario

Merge request reports