# Geo: Improve verification and replication handling of projects without Git repositories ## Problem Statement Geo currently handles projects without Git repositories poorly, causing confusion and inefficiency. Projects without repositories display Project Repositories checksum failure in the UI. Concurrently, the logs in the secondary site report a synchronization failure with the `Error syncing repository: 13:creating repository: cloning repository: exit status 128.`. Also the primary site reports `Repository does not exist` when attempting to checksum those Git repositories. This leads to false error reporting, wasted sync attempts, and unclear Geo health assessments for administrators. ## Workaround Create project repositories on the primary when they don't exist: ```ruby puts "Found #{Project.verification_failed.count} project repos failed to checksum" Project.verification_failed.find_each do |p| puts "#{p.full_path} #{p.ensure_repository.inspect}" end ``` ## Solution ### The endgame **What we need to do:** Moving the git repository replication from the `Project` model to the `ProjectRepository` model. To ensure projects are not replicated and verified anymore, we need: * Update, creation and deletion of Project records do not fire Geo events: * The model does not include `::Geo::ReplicableModel` anymore * No occurrence of `geo_handle_after_update`, `geo_handle_after_create` and `geo_handle_after_delete` is based on the `Project` model class * Update and creation of Project records do not create state records * The model does not include `::Geo::VerifiableModel` anymore * The `project_states` table has been dropped To ensure `ProjectRepository` records are replicated: * Update, creation and deletion of records do fire Geo events, and create state records * The `ProjectRepositoryReplicator` class points towards `ProjectRepository` * Cron jobs won't pick up Project records from the Cron workers, looping through ProjectRepository records instead * `ModelMapper` will not find `Project` to be a valid replicable model, replaced by `ProjectRepository` * The `DataManagement` view does not default to show projects, but project repositories. On the secondary side, the registry is using the new source of truth: * The `ProjectRepositoryRegistry` now belongs to `ProjectRepository` * Its `model_class` and `model_foreign_key` match respectively `ProjectRepository` and `project_repository_id` * All methods from this class which take `project_id` as argument now take `project_repository_id`. This basically means that `project.last_repository_updated_at` becomes now `project_repository.project.last_repository_updated_at` ### How do we get there? One important point is to maintain backward compatibility: if a user switches to the repository-based replication, then we need to ensure they can roll back to project-based replication without data or performance loss. #### High-level plan | | Steps | Completion | |--|-------|------------| | 1 | Introduce a feature flag to be used in the next two steps | :white_check_mark: | | 2 | Insert record in `project_repositories` only when a Git repo is created. | :white_check_mark: | | 3 | Enumerate `project_repositories` instead of `projects` table for verification/replication of Git repos | in progress | | 4 | Add background migrations to delete `project_repositories` rows where Git repos don't exist | planned | After step 3 is completed, when `:geo_project_repository_replication_v2` is enabled, the behaviour of the system should match the endgame. It should be possible to toggle to legacy replication without losing data or re-replicating all project repos. #### Detailed next steps Proposed plan: * **step 1**: ensure under v2, projects are not replicated anymore: * consolidate the event triggers (https://gitlab.com/gitlab-org/gitlab/-/work_items/437929) to make it easier to disable or change them depending on the FF * ensure project events are only triggered under V1 * associate the `ProjectReplicator` to the new model * **step 2**: ensure under v2, the registry can handle `ProjectRepository` records. The `model_record_id` in events consumed for the sync & verification will refer to `ProjectRepository` records, and the `model_foreign_key` needs to be the new FK - this was quite well covered in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/194051+ ## Exit Criteria - [ ] Geo only tracks Git repos for projects that have actual Git repositories ([#546175](https://gitlab.com/gitlab-org/gitlab/-/issues/546175)) - [x] project_repositories table only has records when Git repos exist for a project ([#546176](https://gitlab.com/gitlab-org/gitlab/-/issues/546176)) - [ ] [Documentation explains use of legacy and new version of project repository replication](https://gitlab.com/gitlab-org/gitlab/-/issues/550191) ## Participants - @c_fons - @mkozono - @luciezhao - @dbalexandre ## Development Log ### Previous Status Updates _[This section will be populated with historical status updates as the epic progresses]_ **Date:** 2025-01-06 **Status:** ~"workflow::ready for development" **DRI:** @c_fons ### Current Status Several issues of the epic have been implemented and deployed. The rest are in development or in stages of review. See status update below :point_down: Follow-up epic for a short second iteration of the project has been drafted: https://gitlab.com/groups/gitlab-org/-/epics/19800+ ### Blockers None identified at this time. --- **Start Date:** None **Estimated Due Date:** [TBD] ## Status Update <!-- STATUS NOTE START --> ## Status 2026-03-26 The team decided to pause implementation of the epic, by polishing the remaining issues for future pickup rather than introducing the V1/V2 complexity now. Focus is shifted to bugs until capacity and priorities become clearer. :clock1: **total hours spent this week by all contributors**: 15 :tada: **achievements**: - All sub-issues of the epic have been refined and are ready to pickup :issue-blocked: **blockers**: - none :arrow_forward: **next**: - Refine the remaining issues in the linked [draft follow-up epic](https://gitlab.com/groups/gitlab-org/-/work_items/19800): either close them or fold them back into the current epic _Copied from https://gitlab.com/groups/gitlab-org/-/epics/17974#note_3192661953_ <!-- STATUS NOTE END -->