WIP: Versioning import export PoC

What does this MR do?

Proof of concept related to Rework Import/Export versioning mechanism

Proposal

My proposal is based on similar approach that versionist gem takes for versioning Rest Api.

Version namespace

It’s actually pretty straightforward. We need to namespace our import_export modules with versions that we want to support:

module Gitlab
  module ImportExport
    module V0_2_5 
    end
  end
end

Then move the import_export.yml, import_manager, export_manager, config and code we want to override into gitlab/import_export/V0_2_4, so that we keep with the Rails convention.

gitlab
├── import_export
│   └── V0_2_4
│       ├── import_export.yml
│       └── config.rb
│       └── import_manager.rb
│       └── export_manager.rb
│       └── relation_factory.rb
├── importer.rb
├── relation_factory.rb
├── import_manager.rb
├── export_manager.rb

How does it work

During export, Exporter will read latest version from ImportExpoort::VERSION as before. It will use schema (import_export.yml) and code from corresponding namespace.

During Import, Import will look for export version (It will read VERSION file from exported tar file). If namespace with that version exists, importer will use schema (import_export.yml) and classes from that namespace for importing project. If namespace with that version doesn't exists, importer will return error that version used is not supported anymore...

Backward Compatibility with different versions

Now when we need to provide a non-backwards-compatible version of RelationFactory, we simply create a new module:

module Gitlab
  module ImportExport
    module V0_2_4
      class RelationFactory < Gitlab::ImportExport::RelationFactory
      end
    end
  end
end

In case that we want to be backward-compatible with previous version, we need to create:

module Gitlab
  module ImportExport
    module V0_2_5
      class RelationFactory < Gitlab::ImportExport::V0_2_4::RelationFactory
      end
    end
  end
end

This has its downside. This means that when we stop supporting V0_2_4, this file will have to change. Phasing out V0_2_4 is now no longer a matter of deleting gitlab/import_export/v0_2_4. However, I believe that code quality is something to be considered, we don't want to keep two copies of exactly the same code ...even if the price is a slightly more complex refactor later.

What we can also do

We can write down generators that will allow us to create new import_export version For example: By running

rails generate import_export:copy_version V0_2_4 V0_2_5

this will

• Copy all files from gitlab/import_export/v0_2_4 to gitlab/import_Export/v0_2_5
• Bump version to ImportExport::VERSION to 0.2.5 ...

What is not included by this MR

• Versioning EE modules that are extending regular import export ones.
• Group Export
• Lot of ImportExport code is referencing to Gitlab::ImportExport::[Class]. Since we want to use classes from proper version namespace, we should refactor all references in Gitlab::ImportExport namespace to only [Class], since we want to benefit from proper inheritance.

Conclusion

With this approach we can:

• support different schemes (import_export.yml) for each version.
• have compatibility for multiple versions
• safely remove hacks - like 17799 for mapping merge_request_ids, and the one for renaming relations.
• easily drop support for old version, drop hacks, by simply removing version folder with only small refactor needed**(see my comment above in Backward Compatibility section)
• easily test versions - separate schema and specific code in namespace, similar like we do for EE
• easily support import of older versions. For example 12.0 to 12.5.
• add support to export for specific version. For example in 12.5 we chose as target export 12.0, and we can import in 12.0
• simple rename relation, there is separate schema for each version. (No need to keep double relations costing us resources during import)

Note

Main changes are in Importer, ExporterService, and I introduced ImportManager, ExportManager and VersionManager. Please check them to get an idea.