Resolve "Allow multiple `--config=` inputs, parse from env vars only if `--config=ENV` is set"
Closes #53 (closed)
TL;DR:
This MR adds the ability for CLI arg --config
to accept multiple inputs, including a new --config=ENV
setting which automatically captures settings from environment variables. Breaking change: parse_env_config
no longer defaults to True, and instead must be set explicitly in the CLI or in the class constructor. See "Breaking Changes" below for more info.
Details:
- If one of the supplied inputs is
--config ENV
(or--config=ENV
according to the user's preference), the environment variable parsing rules will be applied to ingest config values from environment variables. - One or more files can also be sent to
--config
. If multiple files are sent, they will be processed in sequential order. - If one or more files conflict for a given setting, the latter provided files will override earlier provided files.
- This behavior allows to you easily "inject" environment overrides by adding
--config=path/to/overrides.json
at the end of the CLI command text.
- This behavior allows to you easily "inject" environment overrides by adding
- If
--config=ENV
is set and one or more files conflict with an environment variable setting, the environment variable setting will always have precedence, regardless of ordering. - One new benefit of this approach is that credentials and other secrets can be stored completely separately from general settings: either by having two distinct config.json files or by using environment variables for secure settings and config.json files for the rest.
IMPORTANT: Breaking Change:
- Prior to this MR, taps automatically collected environment variables regardless of inputs and the Tap constructor sent
parse_env_config=True
by default. Now, the setting must be called explicitly - either by setting--config=ENV
at the CLI or by specifyingparse_env_config=True
in the Tap class constructor.
Edited by AJ Steers