Direct Transfer: Investigate failures of batched migrations

Problem

With the batched migration process, we already noticed a significant performance improvement.

unbatched migration run batched migration run
~1:30hrs 0:30hrs

However, there's also been an increase in failures/errors:

unbatched migration run batched migration run
3 failures 21 failures

image

Note, the batched migration is currently running behind the bulk_imports_batched_import_export feature flag.

Proposal

First, we need to investigate why these failures are happening and then look into resolving them step by step.

tbd.

Edited by Martin Wortschack