[meta] Bidirectional mirroring
A customer has requested bidirectional mirroring to allow their team to move to GitLab because modifications need to be stored upstream (repository external to GitLab) and accept modifications to directly to upstream.
Another example of this being useful is maintaining an open source project in a private company GitLab instance, while simultaneously making it available publicly through a different instance or service. An advanced example of this approach is Facebook, who synchronize work to OSS projects from their internal mono-repo to their public GitHub projects.
A challenge of bidirectional mirroring is the ability to rewrite history. This is a common practice in feature branches, and some teams chose to only permit fast forward merges into master. Although rewriting history is common in feature branches, it is rare in stable branches and is note permitted by anyone for protected branches.
A further challenge is that modifications might be made to both repositories at the same time, or at the very least before repositories could be synchronized. Reducing the window of opportunity for these conflicts to arise should be a priority, because once a conflict arises one or other repositories will require history to be rewritten to resolve the conflict. This should be avoided as much as possible.
Proposal
Use push and pull mirroring simultaneously pointing at the same repository.
Currently both push and pull mirroring are very slow. This increases the likelihood of conflicts. We should make push and pull mirroring faster to reduce this risk.
-
Make pull mirroring faster: API endpoint to trigger pull from push
web hook to avoid polling #3821 (closed) -
Make push mirroring faster: reduce push mirroring delay #3832 (closed) (required #3843 (closed))
When changes are being made to both repositories, any time history is rewritten the sync will fail. This can be avoided by limiting which branches are synced
-
Only sync protected branches -
Pull mirroring option #3844 (closed) -
Push mirroring option #3843 (closed)
-
Future improvements
Although it is possible to activate bidirectional mirroring currently, it is hard because there are significant differences between the push and pull mirroring setup experience, particularly around the use of SSH.
It would be nice to improve this in future: Improved mirroring interface for push, pull and bidirectional mirroring #3959 (closed)
Links / references
- Web hooks:
Documentation blurb
Overview
Bidirectional mirroring allows a repository to be kept in sync between two git servers. It is being developed to provide a migration pathway for large teams from one git server to another, for example from Perforce to GitLab.
An organisation would enable bi-directional mirroring in GitLab. The team can then be incrementally migrated to GitLab without losing the ability to work on the same repository.
Use cases
This is primarily for large customers, and current demand is from a few p4 customers.
Feature checklist
Make sure these are completed before closing the issue, with a link to the relevant commit.
-
Feature assurance -
Documentation -
Added to features.yml