Skip to content

Stream Maps - Aliasing stream still persists original name also

Summary

I need to alias my streams to include a prefix on each so that I dont overwrite an existing table with the same name as the stream. I have an existing issues table from tap-github and want to add the tap-gitlab issues table as well (see squared repo for use case).

I added inline stream maps to alias each stream with a prefix gitlab_issues, this works to create the newly named table but it also continues to persist the original stream as well issues. This seemed like a bug to me but might be expected behavior, I'm not sure.

I also tried to include the "__else__": null at the stream level as described in the docs to hopefully have it exclude those original stream names but ran into a KeyError: 'stream_maps.__else__' - this might be a separate issue so I created one for that too.

  - name: tap-gitlab
    variant: meltanolabs
    pip_url: git+https://github.com/MeltanoLabs/tap-gitlab.git
    select:
    - projects.*
    - users.*
    - merge_requests.*
    - issues.*
    - branches.*
    - commits.*
    - groups.*
    - project_labels.*
  loaders:
  - name: target-athena-gitlab
    inherit_from: target-athena
    config:
      stream_maps:
        projects:
          __alias__: "gitlab_projects"
        users:
          __alias__: "gitlab_users"
        merge_requests:
          __alias__: "gitlab_merge_requests"
        issues:
          __alias__: "gitlab_issues"
        branches:
          __alias__: "gitlab_branches"
        commits:
          __alias__: "gitlab_commits"
        groups:
          __alias__: "gitlab_groups"
        project_labels:
          __alias__: "gitlab_project_labels"
        "__else__": null

Steps to reproduce

meltano, version 1.90.1

What is the current bug behavior?

The original stream is still persisted even after I alias it.

What is the expected correct behavior?

Aliasing a stream would produce only the alias stream and the original would be excluded.

Relevant logs and/or screenshots

Possible fixes

Edited by Pat Nadolny