Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
Meltano
Meltano
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 464
    • Issues 464
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge requests 22
    • Merge requests 22
  • Requirements
    • Requirements
    • List
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI/CD
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • meltano
  • MeltanoMeltano
  • Issues
  • #2208

Closed
Open
Created Jul 30, 2020 by AJ Steers@aaronsteersOwner

Standardize on setting env vars prefixed with plugin name, not namespace or custom `env`

I wonder if the decision to use dynamically generated namespaces, and thereby reducing the amount of config needed per plugin.

In tapdance (maybe similarly in pipelinewise?), the environment variables are scanned at start of execution, and will be automatically included if they begin with f"{plugin_id.replace('-','_').upper()}_", where the plugin_id is something like tap-salesforce or target-snowflake.

So, the below would be automatically parsed by the default conventions:

TAP_SALESFORCE_account   = ...
TAP_SALESFORCE_user      = ...
TARGET_SNOWFLAKE_account = ...
TARGET_SNOWFLAKE_role    = ...

I noticed most of the namespaces in Meltano match this pattern, but there are some odd ones that seem to break the pattern, and I can't see any benefits of doing so. To me, I have to think the "just works" benefit would be higher value than having advanced customizability and then have to rely on memory/lookups to confirm the exceptions to the rule.

From the docs:

# Specify namespace, which will serve as the:
# - prefix for configuration environment variables
# - identifier to find related/compatible plugins
# - default value for the `schema` setting when used
#   with loader target-postgres or target-snowflake
(namespace): tap_covid_19

Thoughts?

Edited Sep 10, 2020 by Douwe Maan
Assignee
Assign to
Fri: Sep 18, 2020
Milestone
Fri: Sep 18, 2020 (Past due)
Assign milestone
Time tracking