[ndjson][optional] create offline tool to convert legacy JSON export archive to ndjson format
Problem to Solve
Issue #197171 (closed) introduces NDJSON for project import.
While long term we want to deprecate legacy JSON export format in favor of ndjson
format, there could be scenarios where legacy JSON still need to be supported:
- A self-hosted customer has a legacy Gitlab version(3 months, or even 6 months older) installed. The customer wants to move to Gitlab.com.
- A customer has a legacy format export file. They regularly use that file to do some testing in their system.
- A customer reported a bug/issue in project import, where it used legacy format JSON export file. Our developer or customer support team wants to reproduce the issue in the new version Gitlab instance. Or, even, the customer wants to try it themselves.
If we encounter the above scenarios after the legacy JSON export format is deprecated in Gitlab, it would be extra steps/effort to convert legacy JSON export archive to the ndjson
format.
For example, let us suppose:
- Gitlab
<= 12.9
only support legacy import/export - Gitlab
13.0, 13.1
support both legacy import/export and ndjson import/export. - Gitlab
>= 13.2
only support ndjson import/export
To import a legacy file to Gitlab 13.2+, it requires:
- find a Gitlab instance with version
13.0
or13.1
, import the legacy project export file - in the same Gitlab instance, export the project into
ndjson
format - import the
ndjson
format archive to desired Gitlab instance(13.2+)
Proposal
An offline conversion tool will make this easier. The tool take the legacy JSON export as input and generate ndjson
format archive. This then can be used to import into a new Gitlab instance.
The tool could be:
- a rake task, or,
- shell script to use
jq
to do the conversion
Besides, we also need to document the usage of the tool.
Alternatively, in case if the script is very simple, we could ignore the tool itself, and just enclose the conversion script in the documentation.