[ndjson][optional] create offline tool to convert legacy JSON export archive to ndjson format

Problem to Solve

Issue #197171 (closed) introduces NDJSON for project import.

While long term we want to deprecate legacy JSON export format in favor of ndjson format, there could be scenarios where legacy JSON still need to be supported:

  • A self-hosted customer has a legacy Gitlab version(3 months, or even 6 months older) installed. The customer wants to move to Gitlab.com.
  • A customer has a legacy format export file. They regularly use that file to do some testing in their system.
  • A customer reported a bug/issue in project import, where it used legacy format JSON export file. Our developer or customer support team wants to reproduce the issue in the new version Gitlab instance. Or, even, the customer wants to try it themselves.

If we encounter the above scenarios after the legacy JSON export format is deprecated in Gitlab, it would be extra steps/effort to convert legacy JSON export archive to the ndjson format.

For example, let us suppose:

  • Gitlab <= 12.9 only support legacy import/export
  • Gitlab 13.0, 13.1 support both legacy import/export and ndjson import/export.
  • Gitlab >= 13.2 only support ndjson import/export

To import a legacy file to Gitlab 13.2+, it requires:

  • find a Gitlab instance with version 13.0 or 13.1, import the legacy project export file
  • in the same Gitlab instance, export the project into ndjson format
  • import the ndjson format archive to desired Gitlab instance(13.2+)

Proposal

An offline conversion tool will make this easier. The tool take the legacy JSON export as input and generate ndjson format archive. This then can be used to import into a new Gitlab instance.

The tool could be:

  • a rake task, or,
  • shell script to use jq to do the conversion

Besides, we also need to document the usage of the tool.

Alternatively, in case if the script is very simple, we could ignore the tool itself, and just enclose the conversion script in the documentation.