Support Deployment Plugins
For Meltano to truly be a 'one-stop-shop' for your data platform, it needs to be able to deploy the individual tools and services that make up that platform. To achieve this Meltano should support deployment plugins, being existing tools for deploying and managing running infrastructure (such as Helm, Terraform, Ansible, Salt, Cloudformation). For those just getting started, Meltano should aim to provide opinionated base config for common scenarios (e.g. containers on Kubernetes), and for power users of these deployment tools the config and internals should still be accessible and easily customisable via existing Meltano config routes.
As a rough guide, the CLI could look something like:
# First a user would add a deployment. This would install all the necessary tooling to actually deploy to this platform during `meltano install`. Meltano will use all of the context of installed plugins, orchestrators and config to create appropriate deployment artefacts.
meltano add deployment --platform kubernetes --environment prod
# Users can optionally configure their deployment
meltano config deployment --environment prod set namespace "team-data"
# Finally a new deployment can be executed 🚀
meltano deploy --environment prod
# Individual tools can also be invoked and used directly
meltano invoke helm list --all-namespaces --all