This project is archived. Its data is read-only. This project is read-only.
Properly handle stream selection and `selected_properties`, including both `RECORD` and `SCHEMA` messages
This would add proper stream selection and property selection via the input catalog (when provided). ## Approach Following from the discussion [here](https://gitlab.com/meltano/tap-base/-/merge_requests/1#note_464222633), the selection filtering should occur at four steps in the process: 1. Selecting or deselecting entire streams. 2. As an input to `get_records()` - so tap developers can avoid collecting data they don't need. - This is available today in some manner, if the developer interrogates `Tap.input_catalog` (not a trivial effort). 3. As a property filter on the standard (base class) transformation from raw data into the RECORD message. - Failsafe for excessive fields being returned from `get_records()`. 4. As a property filter on the standard (base class) generation of SCHEMA messages. - Many taps don't perform this filtering today, which results in downstream columns being unnecessarily (and confusingly) created in the target's destination tables.
issue