Unable to use stream maps for string field.
Summary
Steps to reproduce
I have the following added to my config:
"stream_maps": { "survey_responses": { "email_address": "'string_to_replace'" }
The top level property in the schema looks like:
"email_address": { "type": "string" }
What is the current bug behavior?
The Property that the mapper creates will always be optional When the to_dict() gets called on the property, append_type will always fail here.
What is the expected correct behavior?
append_type shouldn't get called in this case. It works if commented out.
Relevant logs and/or screenshots
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/tap_base.py", line 319, in cli
tap = cls( # type: ignore # Ignore 'type not callable'
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/tap_base.py", line 62, in __init__
self.mapper.register_raw_streams_from_catalog(
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/mapper.py", line 364, in register_raw_streams_from_catalog
self.register_raw_stream_schema(
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/mapper.py", line 427, in register_raw_stream_schema
mapper = CustomStreamMap(
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/mapper.py", line 120, in __init__
) = self._init_functions_and_schema(stream_map=map_transform)
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/mapper.py", line 239, in _init_functions_and_schema
Property(
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/typing.py", line 175, in to_dict
type_dict = append_type(type_dict, "null")
File "/Users/dan.ladd/VSCode/tap-surveymonkey/.venv/lib/python3.8/site-packages/singer_sdk/helpers/_typing.py", line 46, in append_type
raise ValueError("Could not append type because type was not detected.")
ValueError: Could not append type because type was not detected.
Possible fixes
Edited by Dan Ladd