Update import side to be able to import a batch of relations
Once batched relations export is complete we should update BulkImports codebase to support importing of batches instead of a relation as a whole.
See POC !109491 (closed)
On a high-level, we should do the following:
- Add a new import model
BulkImports::BatchTracker
, similar toBulkImports::Tracker
, in order to track import state of a single batch - Add a new import worker, similar to
BulkImports::PipelineWorker
, e.g.BulkImports::PipelineBatchWorker
to perform import of a single batch, instead of a whole relation collection - State of a pipeline tracker should only be marked as finished when all batches are imported, e.g. by adding
FinishBatchedPipelineWorker
as done in the POC.
Edited by George Koltsov