Limit Bulk Import extractors & loaders to 1 each per pipeline
Problem to solve
Currently Bulk Imports Group Migration BulkImports::Pipeline::Runner
has an ability to add multiple extractors
& loaders
.
e.g.
module BulkImports
module Groups
module Pipelines
class MyPipeline
include Pipeline
extractor Extractor
extractor AnotherExtractor
transformer Transformer
loader Loader
loader AnotherLoader
end
end
end
end
Because of that, BulkImports::Pipeline::Runner#run
has to loop through extractors/loaders, increasing it's complexity. In reality, so far (and it might change in the future), there is no need in having multiple extractors or loaders. Usually it's 1 loader and 1 extractor per pipeline.
Proposal
Restrict defining extractors/loaders to 1 each per pipeline in order to reduce BulkImports::Pipeline::Runner#run
complexity. If we are going to need to extract/load from/to multiple sources/destination we can revisit it and consider adding such functionality in the future, but there is no need in having it now.