Improve test coverage
Problem to solve
We now rely on manual pipeline invocation to confirm that a change doesn't break any pipelines. This is not ideal for many reasons and as a result, many changes breaks our pipeline in some unexpected way.
This issue aims at improving the test coverage of promptlib, so every bug can be catched early.
Proposal
Rough plan:
- Add more end-to-end tests for all pipelines.
- Add more end-to-end tests for all different configurations of our pipelines. For example, read from csv and read from rake output.
- Add more unit tests for individual Beam pipelines and functions.