Geo 17.11 Outlook

This is the current outlook for the 17.11 milestone. The Geo team schedules issues and works in a continuous Kanban manner, while still aligning with milestones. See our process page for more details.

Last Updated: 17th March 2025

Org mover MVC

Cells and Organizations are foundational components of GitLab's next-generation architecture of GitLab.com.

The Cells architecture is horizontally scalable, resilient, and provides a more consistent user experience.

As we approach Cells 1.5, we need a way to migrate organisations from the current GitLab.com infrastructure to Cells. We need to achieve this with minimal disruption to our users on GitLab.com. The org mover will help us achieve this by replicating the organisation's data to a Cell. The org mover will perform the replication in the background while users of the organization continue to use GitLab. Any new changes made by users will also be synced to the Cell.

Once the data between the current GitLab.com infrastructure and the Cell are in sync, we will cut over the organization to the cell. The cutover may initially require some downtime to allow the replication to catch up.

• Swap FK projects.pool_repository_id to pool_rep... (gitlab-org/gitlab#521717 - closed) Douglas
• Add sharding key for `lfs_objects` (gitlab-org/gitlab#490482 - closed) - Douglas/Mike

Org Mover Design priority2

Who: Douglas/Mike

We are currently discussing the Org Mover Design, testing/investigating with a POC and also planning to update the design docs and add more documentation on how Org Mover works at a high level.

• Org Mover: POC to replicate non-PG data in GDK + Cells - Douglas
• Org Mover: Design how to handle shards and its dependent tables across a move - Douglas
• Org Mover - Decide how to handle data on object... (gitlab-org/gitlab#520191) - Mike/Douglas
• Org Mover: Update design doc with diagrams and more design details - Mike

Begin rollout of checksumming of all data on GitLab.com priority2

Who: Scott, Natanael

A pre-requisite for org mover is to checksum data on the source cell. The first source cell will be the GitLab.com monolith. Based on experience with Geo, we know that we will discover orphaned database records and files that will need resolving prior to being able to successfully move and org. The progressive rollout of checksumming for data types on GitLab.com will allow us to investigate orphaned records and correct them well ahead of any migration. We also will be limiting the checksumming by a selective set of organizations. To fully complete this feature, this has dependencies on sharding work being completed.

• Org Mover - Move all verification details into separate tables - Scott
• Partition table ci_job_artifact_states - Natanael

Geo Improve Observability priority2

Who: Zack/Chloe/Aakriti

In order to support Cells and Org Mover, current Geo functionality for replication and verification especially surfaced in the UI needs improvement and better observability. The goals we want to achieve with this epic are:

• Enrich existing interface

  Why: This will provide immediate visibility into errors for the systems admin without needing to resort to the rails console. Since this is anticipated to be a relatively small effort we will tackle this first.

  • Geo Replicables: Enhance replicable lists information
    • Geo: Improve layout of tabs in replicables list... (gitlab-org/gitlab#410401 - closed) - Zack
    • Geo Replicables List: Store query in the URL (gitlab-org/gitlab#515011 - closed) - Zack
    • Geo Replicables API: Add query `arguments` to t... (gitlab-org/gitlab#520345 - closed)- Chloe

• Improve primary verification experience

  Why: This will provide immediate visibly into verification errors on the primary site without needing to resort to the rails console. We've noticed an increasing need for this visibility amongst new Geo customers. This will also help improve the first time experience.

• Level-up available actions to fix problems

  Why: This will reduce the need to go to the rails console to apply specific actions to a sub-set of objects. We can build on this foundation to provide mode repair actions in future.

• Geo: Correlate logs across sites (gitlab-org&16469)

  Why: This will make it a lot easier to find relevant logs quickly when debugging to allow for better observability across the secondary and primary in correlating logs together.

  • Geo: Add correlation ID when a secondary site p... (gitlab-org/gitlab#513035) Aakriti
  • Geo: Add correlation ID to Geo::RequestService ... (gitlab-org/gitlab#513034) Aakriti

Misc

• Fresh GEO instance not working with 15.10 - sol... (gitlab-org/gitlab#404016 - closed) priority2 - Chloe
• https://gitlab.com/gitlab-org/gitlab/-/issues/520124+ priority2 - Natanael
• database major upgrade - make promote_database in pg-upgrade.rb idempotent priority3 - Natanael
• GraphQL API: Deprecate Geo registries keyword arg (gitlab-org/gitlab#517791) - Mike
• Geo: Investigate the impact of deprecating thre... (gitlab-org/gitlab#525508) - Scott

Customer Support Triage DRIs: Aakriti & Chloe