meltano.yml does not allow referencing arbitrary environment variables from .env
Based on my understanding of meltano.yml
, I would expect this to work:
.env
:
WAREHOUSE_URL=postgres://localhost:5432/meltano
meltano.yml
:
plugins:
loaders:
- name: target-postgres
pip_url: git+https://github.com/meltano/target-postgres.git
config:
url: $WAREHOUSE_URL
However, with this configuration, this is the result:
$ meltano config target-postgres
[2020-07-15 12:23:01,556] [84379|MainThread|root] [WARNING] Variable WAREHOUSE_URL is missing from the environment.
{"url": null, "user": "warehouse", "password": "warehouse", "host": "localhost", "port": 5502, "dbname": "warehouse", "schema": null}
My hunch is that load_dotenv
isn't called before meltano.yml
is parsed.
This is not a blocker for us, the problem goes away if PG_URL
is used instead, or if export WAREHOUSE_URL
is used before meltano config
. Just wanted to bring it up because I think most would find the behavior surprising.
I didn't add the bug labels because I'm not sure if this is actually a bug or if this behavior isn't supported.