Custom topics do not show up in Meltano UI
What is the current bug behavior?
If you add a custom topic together with the default topics for a tap, it is not showing up in Meltano UI.
This bug breaks the Advanced Tutorial - Adding Custom Transformations and Models which was working before meltano add model
was introduced.
What I did:
-
Created a new project
-
Added default SFDC model by using
meltano add model model-salesforce
-
Added Custom topic by following instructions in the Advanced Tutorial - Adding Custom Transformations and Models.
-
Started Meltano UI, selected the new project and made sure that the topics were compiled.
The important part here is that there is a topic that compiles properly (sfdc_custom.topic.m5oc):
$ ls -al model/ -rw-rw-r-- 1 iroussos iroussos 646 Απρ 10 19:06 sf_account_category.table.m5o -rw-rw-r-- 1 iroussos iroussos 458 Απρ 10 19:16 sfdc_custom.topic.m5o -rw-rw-r-- 1 iroussos iroussos 3126 Απρ 10 19:17 sfdc_custom.topic.m5oc -rw-rw-r-- 1 iroussos iroussos 2932 Απρ 10 19:17 sfdc.topic.m5oc -rw-rw-r-- 1 iroussos iroussos 1983 Απρ 10 19:04 sf_opportunity_won.table.m5o -rw-rw-r-- 1 iroussos iroussos 56 Απρ 10 19:17 topics.index.m5oc
-
In meltano UI I only get to see the topic added by
meltano add model model-salesforce
-
Also, the topics.index.m5oc only has the default, prepackaged model that comes with
meltano add model model-salesforce
:{"sfdc": {"designs": ["sf_opportunity_history_joined"]}}
What is the expected correct behavior?
Users should be able to add custom transforms and models and see the results in Meltano UI.
We should be able to at least complete the Advanced Tutorial - Adding Custom Transformations and Models.
Steps to reproduce
- Create a new project
- Add default SFDC model by using
meltano add model model-salesforce
- Add Custom topic by following instructions in the Advanced Tutorial - Adding Custom Transformations and Models.
- Start Meltano UI, select the project you created and check the available designs --> The custom topic and its designs are missing.
You can also quickly check model/topics.index.m5oc
and see that the custom topic is not there.
Possible fixes
If you can, link to the line of code that might be responsible for the problem or suggest a fix
Further regression test
Ensure we automatically catch similar issues in the future
-
Write additional adequate test cases and submit test results -
Test results should be reviewed by a person from the team