Hub metadata for plugins should be a superset of necessary fields in discovery.yml
As a path to unblock Add lock artifacts during `meltano add` for sta... (meltano#3031 - closed) and also to replace discovery.yml in the Meltano project with the metadata here in the Hub, we'd like to ensure that Hub metadata is a superset of the needed data that is currently present in meltano.yml.
Important exceptions and deprecations
| Setting | Description | Resolution or Mitigation |
|---|---|---|
settings.env |
A method of either injecting or sourcing environment variables | We support for non-singer plugins. We remove those references used for "sourcing". We remove injection scenarios for Singer plugins unless required (logging an issue with the dev requesting them to support a settings.json - based config). We may also consider a rename to something like as_env_var or mapped_env_var so this is more specifically targeted to injecting env vars rather than its current dual role of injecting and/or sourcing. |
settings.env_aliases |
Same as env but allowing multiple inputs and outputs. |
Deprecate this. No support. |
settings.select |
Used to create default select patterns for a given tap. | Deprecate. This was primarily needed for transform packages (also deprecated) which had specific table and field requirements. |
settings.metadata |
Used similarly to select as a means of overriding selection defaults. |
Deprecate, same as select metadata. |
settings.aliases |
Alternative names that can be used in meltano.yml and with meltano config set. |
Keep as-is and provide a list of common/standard aliases to support interop across forks and utilities: e.g. password, user, database_name, schema_name, etc. |
settings.value |
The default value if no other value is provided. | Keep, but perhaps rename to default to make the function of this more clear. |
settings.placeholder |
Used to hint to the user what kind of entry is supported | Keep (as optional). |
settings.description |
Optional. Used to provide inline context. | Keep. |
settings.tooltip |
Optional. Used to provide additional inline context. | Keep. Or remove and merge with description. |
settings.documentation |
Optional. Used to provide inline context. | Deprecate and merge into description. |
settings.value_processor |
Modify value before loading it from source. | Deprecate. |
settings.value_post_processor |
Modify value after loading it from source. | Deprecate. |
settings.protected |
Use in combination with value to provide an uneditable default. |
Deprecate. |
Move to a single yaml format in the hub per plugin
Currently we have two yaml files per (discoverable) plugin. This would move us to one-yaml-per-plugin and would allow us to fully deprecate discovery.yml after a certain amount of time has passed.
Alignment of Settings kind
As discussed in Standardize setting types including sensitive a... (#25 - closed), our settings types need to be converted or aligned between Meltano and the hub.
A possible path forward:
- Decide in #25 (closed) what the new settings types/kinds will be, probably including a separate "sensitive" boolean flag per setting.
- Add support for those new settings types in Meltano.
- Deprecate, but do not entirely drop support for the old settings types. (Essentially, those would be aliases until we fully remove them.)
- Consider deprecating the older style in Meltano in a
2.0release.
Edited by AJ Steers