'A pipeline for data source [MY_CUSTOM_TAP] has not run yet' error when accessing report builder in Meltano UI
Steps to reproduce
Add a custom tap (whose definition doesn't appear in the discover.yml file) and then after running a pipeline for this tap, then go to the Report builder section in the Meltano UI. The following error will appear in a red error message box: A pipeline for data source [MY_CUSTOM_TAP] has not run yet. Please set up your connection.
Possible fixes
find_definition_by_namespace
(https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin_discovery_service.py#L250) is called from the the find_namespace_schedule
method of ScheduleService
(https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/schedule_service.py#L135). However, this doesn't find a plugin definition in this case due to it being added as a custom extractor and its definition not been added to the discovery.yml file.
An easy fix might be to just add this custom definition to the discovery.yml file. However, in this particular case, the extractor is unlikely to be used by many other people, so perhaps it doesn't make sense to add it to the public Meltano discovery.yml. Instead, I could just use a local discovery.yml
file.
An alternative would be to change the code so that it can discover custom plugins by inspecting meltano.yml
. What are your thoughts on the best approach?