Partial replication of Gitlab.com to another Geo location

Problem to solve

Currently, Gitlab.com is hosted in the US and no Disaster Recovery or Geo replication is available in other geographic areas (there are East coast and West coast deployments, I believe). This means, that in case of a disaster affecting Gitlab.com in the entire US, no DR recovery is possible from another geographic area.

However, creating a Geo node for Gitlab.com faces several challenges:

  • Scale - Gitlab.com is deployed at a large scale for millions of users
  • Cost - Running full replication is expensive (due to the infrastructure required)
  • Complexity - managing another Gitlab.com Geo node increases the administrative burden for infrastructure

Intended users

  1. In case of a full mirror this could be used as (another) DR solution for GitLab (dogfooding!)
  2. Non-US customers

Further details

Read-only replicas for Gitlab.com are not available outside the US and non-US customers could potentially benefit from a local Gitlab.com Geo node. One of the main challenges is the scale of the Gitlab.com deployment and some differences between Gitlab.com and a smaller on-premise customer installation.

There are a number of current limitations in Geo selective sync as well that may need to be tackled.

Proposal

  1. Create a non-US Geo node for Gitlab.com e.g. in the EU
  2. Sync a subset of projects based on customer demand using Geo selective sync. This could be used for gold/platinum customers. This would also reduce the complexity/cost.
  3. The Geo node could be extended gradually to include more projects

Permissions and Security

  • Customer data would be copied from the US to another location. There may be data compliance issues.

Documentation

Testing

This would require extensive testing in order to be a viable DR solution

What does success look like, and how can we measure that?

Success metrics:

  • Percentage of projects replicated successfully to Non-US instance e.g. 5% of customer projects or on demand

Acceptance criteria:

  • A non-US Gitlab.com Geo node that replicates a subset of projects from the US
  • A non-US instance can be used for Geo replication
  • A non-US instance can be used as DR solution for dedicated customers

What is the type of buyer?

  • Enterprise Gold/Platinum

Links / references

https://docs.gitlab.com/ee/administration/geo/replication/configuration.html#selective-synchronization

Edited by Fabian Zimmer