Skip to content

Added end-to-end tests for all pipelines

Hongtao Yang requested to merge hyang/end-to-end-tests into main

What does this merge request do and why?

End-to-end tests for all pipelines.

This MR is the first iteration of end-to-end tests. In this MR, we execute the whole pipeline in tests, so it is easier to catch bugs.

In this first iteration, the tests will call third-party APIs and will read from and write to BigQuery tables. Because of this, we do not run the end-to-end tests in CI. In later iterations, we should think about mocking those components so that the tests doesn't depend on other APIs and credentials.

Ref: #205

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Hongtao Yang

Merge request reports