Skip to content

Geo: Implement snippet repository replication for organization migration

Problem

As part of organization migration from Legacy Cell to Protocell, the Geo team needs to implement special handling for snippet repository replication. The snippet_repositories table requires sharding implementation, schema changes, and custom migration logic to handle organization migration properly.

Background

snippet_repositories are being sharded by organization_id (currently marked severity::4).

Proposal

  • Replace shard_id FK with LFK (loose foreign key)
  • Classify as gitlab_main_org (it is currently gitlab_main_cell_local)
  • Define sharding key as group_id. It is already non-nullable.

On migration:

  • Copy snippet_repositories rows to the Protocell according to sharding key (as usual). This will be done as part of normal PG replication.
  • When copying snippet_repositories, exclude/nullify shard_id values (since they are invalid). If using AWS DMS for PG replication, use a transformation.
  • Geo can sync snippet Git repos as usual, with one addition: If shard_id is NULL, then set it with ::Repository.pick_storage_shard
  • Ensure there are sufficient automated (unit? QA?) tests to meet GitLab's standards
  • Smoke test locally

Related

Edited by Michael Kozono