This project is archived. Its data is read-only. This project is read-only.
Duplicate pipelines in Meltano UI after Extractor fails
While testing meltano/meltano#1165 (check the [comment thread here](https://gitlab.com/meltano/meltano/issues/1165#note_284221095)), I got a very weird bug with the pipeline for the Gitlab Extractor being duplicated after the extractor failed: ![image](/uploads/a83f6f3b1ef1d75c582fd8155db14a39/image.png) What I did: 1. Initialised a new project 1. Added a run the Gitlab Extractor with a wrong Group name --> It failed ![image](/uploads/7a3bc3002287b917bfe885174aedd2c9/image.png) 1. Run the Extractor again with a wrong Project name --> It failed again Somewhere between step 2 and 3 or immediately after step 3, the new pipeline above was created in the UI: 1. It has the Gitlab Name 1. It shows as succeeded 1. It has no time on when it finished 1. It has no logs of course (loading forever when I click on the log option) Of course no pipeline has succeeded and if you click on the Reports option on either one of the pipelines, the models are still grayed out. This seems like a pure UI bug as there is no additional pipeline reported in our logs: ``` sqlite> select * from job; 1|pipeline-1581077096062|FAIL|2020-02-07 12:04:58.583928|2020-02-07 12:05:02.078277|{"error": ""}|0|d4cd238d0e434ceca12124cc9d223710 2|pipeline-1581077096062|FAIL|2020-02-07 12:10:36.132317|2020-02-07 12:10:43.505225|{"error": ""}|0|e94b4edb4757461c8d62e9fe83d2074e ``` But it is not, as there is an entry for the new pipeline in `meltano.yml` ``` ... ... ... project_id: 0a235f0f-e068-46a7-8e2d-f26c74f30245 schedules: - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581077096062 start_date: 2020-01-31 22:00:00 transform: run - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581077433858 start_date: 2020-01-31 22:00:00 transform: run ``` And when I restart . Meltano UI, the additional pipeline is still there. I can start it manually and then it behaves as a normal pipeline, so I now have 2 pipelines :-) Or should I make them 3? Yep, If I try to fix the connection and re-run the Tap, the first (bottom) pipeline runs again and a new one is added on top. And I can go like that and keep on adding more pipelines: ![image](/uploads/97ab43240d7a60211291e7e6b99cc190/image.png) ``` schedules: - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581077096062 start_date: 2020-01-31 22:00:00 transform: run - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581077433858 start_date: 2020-01-31 22:00:00 transform: run - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581078835942 start_date: 2020-01-31 22:00:00 transform: run - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581078932599 start_date: 2020-01-31 22:00:00 transform: run - extractor: tap-gitlab interval: '@once' loader: target-postgres name: pipeline-1581079022182 start_date: 2020-01-31 22:00:00 transform: run ``` So, to recap: Every time I try to fix the connection and restart, one more pipeline is added on top of everything else.
issue