Standardize the database connection
Databases should be treated as a Meltano concept, typically defined as an entry in the
As Meltano uses SQLAlchemy behind the hood, it is natural to use the
dialect[+driver]://user:password@host/dbname[?key=value..] scheme as a naming convention across all database connections.
databases: - name: warehouse namespace: sqlite settings: - name: dialect value: sqlite - name: user value: warehouse - name: password value: warehouse - name: host value: 'localhost:5432' - name: dbname value: warehouse
- Refactor database connection as a Meltano plugin
- Refactor the model connection to use the Database definition
- Generate loader configuration using
Databases, via the
kind. For instance,
target-sqlitecould have these settings:
name: target-sqlite namespace: sqlite settings: - name: database kind: sqlite
We'll have to think carefully where we want the mapping (my guts tells me
namespace => kind) to happen, but this would alleviate much of the UX pain for the loaders.
- Expose the meltano system database?