This project is archived. Its data is read-only. This project is read-only.
Handle unsupported metadata in singer taps
<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "bug" label and verify the issue you're about to submit isn't a duplicate. If you are submitting an issue with a tap, please include: - account details - target details - entities selected with meltano select (if you have selected any entities), as the bug may be related to a specific entity - the full elt command you are running - full output of the meltano elt command. Logs can get pretty long, so you can add the full log as a snippet in the Meltano project and add a link in the issue. ---> ### What is the current *bug* behavior? Meltano currently (v1.39.1) ignores the metadata with `"inclusion": "unsupported"` and try to include such attributes for an object causing the tap to fail. ### What is the expected *correct* behavior? Do not include metadata with `"inclusion": "unsupported"` and conform with the Singer specification at https://github.com/singer-io/getting-started/blob/master/docs/DISCOVERY_MODE.md#metadata ### Steps to reproduce _How one can reproduce the issue?_ This issue can be seen with tap-zuora. Add custom extractor tap-zuora and run `meltano elt tap-zuora target-csv`. ### Relevant logs and/or screenshots ``` CRITICAL There is a syntax error in one of the queries in the AQuA input (Query = select CreatedById, CreatedDate, EndDate, ErrorFileUrl, Id, ProcessedAccounts, ProcessedCharges, ResultFileUrl, RunNumber, StartDate, TotalAccounts, UpdatedById, UpdatedDate, Entity.Id from ChargeMetricsRun where UpdatedDate >= '1900-01-01T00:00:00 (+00:00)' order by UpdatedDate asc, Error = There is no field named EndDate) ``` For EndData the metadata produced by discovery is marked as unsupported: ``` { "breadcrumb": [ "properties", "EndDate" ], "metadata": { "inclusion": "unsupported" } } ``` ### Possible fixes Ignore all unsupported fields when sending a query to tap ### Further regression test _Ensure we automatically catch similar issues in the future_ - [ ] Write additional adequate test cases and submit test results - [ ] Test results should be reviewed by a person from the team
issue