Mark group import/export to be stable (currently it is experimental) with ndjson support
TL;DR
With the %13.0 and group import/export using ndjson almost finished. We have great opportunity to drop old format and switchover to ndjson one.
This allows us to mark this feature as stable (from experimental) and consider the major performance issues to be solved #199037 (comment 279647849).
Problem to solve
The group import/export
is a new feature (2-3 milestones old), and when it was shipped it was mark as subject to change. When doing that we already did know that this is unperformant and we gonna have to fix it sooner (#199037 (comment 279647849)).
The Project import/export already today is not defining forward and backward compatibility, where the forward compatibility is pretty much broken, as we continue adding new models.
The Group on the other hand has the same design problems: it does not define forward/backward compatibility, so, it I would assume that this not functional if you don't use matching version of GitLab. We build a safeguard into Group Import/Export to say Introduced in GitLab 12.8 as an experimental feature. May change in future releases. to allow us to change the format, as we knew at a time of introduction that this is suboptimal.
Given the:
- limited users (I'm aware at this moment of only our GitHost customer)
- feature marked as experimental
- format of data being inefficient
- making it troublesome to support old format
- %13.0 being a new major milestone
I would say that:
- it seems fully reasonable to use a new and efficient data format (ndjson)
- mark this as a final format
- do not provide backward compatibility for something that was experimental
- with introduction of ndjson mark this as stable
Proposal / Documentation
We do miss a proper documentation for Group Import/Export that would define compatibility, kind of similar to https://docs.gitlab.com/ee/user/project/settings/import_export.html.
I believe we should create a user-guide for Group Import/Export documenting clearly what can be imported where.
I would assume based on past experience working Project Import/Export that we should be careful how wide we define this compatibility. I think we should create for %13.0 similar document to this one, with the following expectations:
- we allow to import exported groups only in GitLab newer or equal (export from %13.0 can be imported into %13.0, %13.1, and so on): this is something that we test today as part of our testing suite
- we do not allow to import exported groups if it was created be newer version (export from %13.4 cannot be imported into %13.0): we cannot test that, as we do not actively develop the older version of GitLab to adapt and support newer structure of data
The practice says that our Support team always when working with import/export functionality requests to use matching versions of GitLab to avoid problems with data migrations, schema mismatches, renames and new structures.