Skip to content
GitLab
Next
    • GitLab: the DevOps platform
    • Explore GitLab
    • Install GitLab
    • How GitLab compares
    • Get started
    • GitLab docs
    • GitLab Learn
  • Pricing
  • Talk to an expert
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
    Projects Groups Topics Snippets
  • Register
  • Sign in
  • Meltano Meltano
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
    • Locked files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • MeltanoMeltano
  • MeltanoMeltano
  • Issues
  • #2364
Closed
Open
Issue created Sep 28, 2020 by Douwe Maan@DouweMOwner

Add support for plugins using `BATCH` message type

Once Meltano supports the PipelineWise taps and targets out of the box (#2357 (closed)), it would be great if we could also support Fast Sync when meltano elt is run with a supported pair of PipelineWise taps and targets:

Fast Sync is bypassing the Singer Specification for optimised performance. Primary use case of Fast Sync is initial sync or to resync large tables with hundreds of millions of rows where singer component would usually be running for long hours or sometimes for days.

See:

  • https://transferwise.github.io/pipelinewise/concept/fastsync.html
  • https://transferwise.github.io/pipelinewise/concept/replication_methods.html#fast-sync
  • https://github.com/transferwise/pipelinewise/tree/master/pipelinewise/fastsync
Edited Nov 29, 2021 by Taylor A Murphy
Assignee
Assign to
Time tracking