Default transferwise variants of tap-postgres and tap-mysql are incompatible with default datamill-co variants of target-postgres and target-snowflake
As found in #2462 (closed):
target-postgres | psycopg2.ProgrammingError: syntax error at or near "-" target-postgres | LINE 1: CREATE TABLE tap_postgres.public-account_group ("code_prefix...
This indicates that tap-postgres (https://meltano.com/plugins/extractors/postgres.html, https://github.com/transferwise/pipelinewise-tap-postgres) is including the schema (
public
) in the stream name it passes to the target (public-account_group
), while target-postgres (https://meltano.com/plugins/loaders/postgres.html, https://github.com/datamill-co/target-postgres) expects the stream name to just be the table name (account_group
), and it breaks when it sees the-
.I suggest using the
transferwise
variant of target-postgres instead: https://meltano.com/plugins/loaders/postgres--transferwise.html, https://github.com/transferwise/pipelinewise-target-postgres, which was built to be used with the tap-postgres you're currently using, and does support schemas in stream names.
We can start by mentioning this limitation under https://meltano.com/plugins/extractors/mysql.html#next-steps and https://meltano.com/plugins/extractors/postgres.html#next-steps.