Improved developer experience with docker-compose ft. massive refactor
Take two of !52 (merged)
I made a screencast that goes over all these changes and the thought process behind them:
- Screencast: https://drive.google.com/file/d/1rGPiT_597YdKnR0SDB03fmW6_QWvw8ZE/view
- Screencast summary: https://drive.google.com/file/d/1rGPiT_597YdKnR0SDB03fmW6_QWvw8ZE/view?t=1120
UPDATES
See these two comments about running Meltano outside of docker
- See this comment !58 (comment 103989476)
- Also see this comment !58 (comment 104193747)
High level
- Booting a demo version of the app is extremely simple. Down to three commands: https://gitlab.com/austinpray/meltano/blob/efd418e13f7504467599d80740b1d003afd601bd/README.md#how-to-install-and-run-meltano
- The build process uses Make so the build command api is super simple. Just run
make
and only the parts of the app that have changed will be rebuilt. - Improvements to the production docker setup. This dev docker-compose setup uses images that are deployable to production. Dev like you prod.
- Meltano is now pip-installable.
- Meltano uses the python packaging system to resolve plugins and stuff instead of a bespoke system
- Removed a ton of assorted jank such as
- python coding errors
- misunderstandings of python's import and module system
- merge conflicts that were committed to master
- files that were gitignored but checked into the repo
If you want to know more about what commands are available to orchestrate the dev environment you can just look at the Makefile or run make explain_makefile
and then the following will be available in your browser:
Next Steps
We need to review again and iron out how this plays into the overall meltano strategy.
Edited by Melty Bot