Offline Direct Transfer migrations architecture design document
Why is this change being made?
This MR adds an architecture design document for offline Direct Transfer migrations. It's not entirely complete, but this is the basis of what offline imports could look like, and in the future, how customers can import from anywhere. Some specific sections that will need further iteration include:
- Exported file structure and metadata file structure will need to be outlined in detail.
- Import-side changes will also need to be outlined in more to describe how all non-file-based pipelines file-based pipelines can import file-based imports when there's no connection to the source.
- Specify the earliest version of GitLab that can be exported with Congregate.
- Export-side changes necessary to support an API and UI within GitLab itself for new enough source versions.
An architecture design document seemed warranted because this involves significant changes to Direct Transfer, it requires creating a standard format for files imported by Direct Transfer, and involves changes to Congregate that support offline migrations.
References
- Discovery issue: Discovery - Support automated migrations betwee... (gitlab-org/gitlab#517490 - closed)
- Offline migrations epic: Support semi-automated migrations between offli... (gitlab-org&8985)
- Congregate documentation
- Architecture design workflow handbook page
Author and Reviewer Checklist
Please verify the check list and ensure to tick them off before the MR is merged.
-
Provided a concise title for this Merge Request (MR) -
Added a description to this MR explaining the reasons for the proposed change, per say why, not just what - Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
-
Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI) - If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the
Maintained bysection on the page being edited - If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
- The when to get approval handbook section explains the workflow in more detail
- If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the
-
For transparency, share this MR with the audience that will be impacted. -
Team: For changes that affect your direct team, share in your group Slack channel -
Department: If the update affects your department, share the MR in your department Slack channel -
Division: If the update affects your division, share the MR in your division Slack channel -
Company: If the update affects all (or the majority of) GitLab team members, post an update in #whats-happening-at-gitlab linking to this MR - For high-priority company-wide announcements work with the internal communications team to post the update in #company-fyi and align on a plan to circulate in additional channels like the "While You Were Iterating" Newsletter
-
Commits
- Added overview of offline migrations in new architecture design document
Edited by Sam Word