Add option to modify schema properties of a stream in meltano.yml
Meltano currently supports modifying only the metadata section of streams in the underlying
catalog.json. We should be able to modify the attributes under the schema section as well. This would enable users to overwrite type constraints when a taps
--discover method doesn't play well with a target, and the user is confident in the type of data points that exists in the source data.
I am currently using
tap-google-sheets together with
anyOf statements for numeric columns, which isn't supported by
target-snowflake. When using singer, I modify the columns that use
anyOf types in the jsonschema with an appropriate datatype, such as integer or float. With meltano, my hands are tied.
Ideally, these incompatibilites would be handled either at the source dataset, or by a robust tap. But taps can often be wrong/incompatible, and having the option to overwrite parts of catalog seems like beneficial.