Standardize on setting env vars prefixed with plugin name, not namespace or custom `env`
I wonder if the decision to use dynamically generated namespaces, and thereby reducing the amount of config needed per plugin.
In tapdance (maybe similarly in pipelinewise?), the environment variables are scanned at start of execution, and will be automatically included if they begin with f"{plugin_id.replace('-','_').upper()}_"
, where the plugin_id is something like tap-salesforce
or target-snowflake
.
So, the below would be automatically parsed by the default conventions:
TAP_SALESFORCE_account = ...
TAP_SALESFORCE_user = ...
TARGET_SNOWFLAKE_account = ...
TARGET_SNOWFLAKE_role = ...
I noticed most of the namespaces in Meltano match this pattern, but there are some odd ones that seem to break the pattern, and I can't see any benefits of doing so. To me, I have to think the "just works" benefit would be higher value than having advanced customizability and then have to rely on memory/lookups to confirm the exceptions to the rule.
From the docs:
# Specify namespace, which will serve as the:
# - prefix for configuration environment variables
# - identifier to find related/compatible plugins
# - default value for the `schema` setting when used
# with loader target-postgres or target-snowflake
(namespace): tap_covid_19
Thoughts?