Tap and target invocation errors show up in `meltano elt` output but not in logs and UI
What is the current bug behavior?
In the current latest Meltano version (1.58.0), the error message which is raised when a tap has an invalid catalog file is output when run on the CLI via meltano elt my-tap my-target
, but the same error message is not shown in the log modal window when run in the UI.
What is the expected correct behavior?
All error messages should be shown in both the CLI and the Meltano UI.
Steps to reproduce
-
One way to reproduce this is to edit the
core/plugin/singer/tap.py
file to force an error to be raised. For example, simply add afinally
block below line 270 in https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin/singer/tap.py#L270 which will raise an error (for example, a PluginExecutionError can be raised here). -
When running
meltano elt my-tap my-target
on the command line, the error will now be raised and the message will be output. -
However, if the same tap/target combination is then run by executing a pipeline via the Meltano UI, the above error message does not appear in the modal window which shows the log.
Possible fixes
A simple fix would be to use a logging.error
message in the except block in tap.py. I haven't investigated if other types of errors are not being surfaced in the Meltano UI, so this simple fix might not be sufficient.