Skip to content

Add a CI pipeline

I think we can add a meltano.yml to this project that comes with:

  1. orchestrator airflow
  2. a file bundle with pip_url: . (so that it always uses the most recent version from the current branch)
  3. some schedules

Then we can add a .gitlab-ci.yml that runs meltano invoke airflow scheduler and validates the expected output, i.e. there should be no error about meltano.py, or the expected lines confirming that a DAG was created for each schedule.

Similar to how https://gitlab.com/DouweM/tap-investing tests itself using https://gitlab.com/DouweM/tap-investing/-/blob/master/meltano.yml and https://gitlab.com/DouweM/tap-investing/-/blob/master/.gitlab-ci.yml#L18.