Export in batches during GitLab migration

Problem

The relation export job of CI pipelines often gets interrupted during GitLab migration.

Proposed solution

Let's look into changing the way we export, to export in batches, means more archive files. Instead of having 1 massive tar.gz file we'd have smaller batches.

This solution would reduce the interruptions and the required disk space to export relations.

Perhaps, we can use the source instance version to know whether the batch export is supported or not.

Edited Oct 11, 2022 by Magdalena Frankiewicz