Geo: Enumerate project_repositories instead of projects for verification/replication of Git repos

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Overview

This issue is part of a multi-step implementation plan to handle Geo verification/replication of projects without Git repositories, reducing errors caused when Gitaly tries to fetch non-existent repos.

Like Design Management Repositories, we want Geo to enumerate the project_repositories table instead of the projects table, because a ProjectRepository record represents an actual Git repo, not a Project itself (in the GitLab Rails code). This will simplify the logic for determining when a Git repo needs to be fetched for verification/replication.

This is the core of V2 project repository replication, gated behind the geo_project_repository_replication_v2 feature flag.

What needs to be done

The branch 546175-geo-enumerate-project_repositories-instead-of-projects contains a proof-of-concept that is not yet mergeable. The following changes are proposed there and need to be completed:

ProjectRepositoryReplicator (ee/app/replicators/geo/project_repository_replicator.rb)

  • model is gated to return ::ProjectRepository when the FF is enabled, ::Project otherwise.
  • should_publish_replication_event? correctly gates events to ProjectRepository records only under V2.
  • repository, pool_repository, and object_pool_missing? are delegated through model_record which can now be either a Project or a ProjectRepository.

Geo::ProjectRepositoryRegistry (ee/app/models/geo/project_repository_registry.rb)

  • A belongs_to :project_repository association is added alongside the existing belongs_to :project.
  • model_class and model_foreign_key are FF-gated to return ProjectRepository / :project_repository_id under V2.
  • A model_accessor helper is introduced to return the right model record depending on the FF.
  • before_validation and after_find callbacks call populate_missing_foreign_keys to keep both FKs in sync during the V1/V2 migration period.

EE::ProjectRepository (ee/app/models/ee/project_repository.rb)

  • pool_repository, object_pool_missing?, and last_repository_updated_at are delegated to project, so that ProjectRepository can be used as a drop-in for Project in the replicator.

Factories and specs

  • spec/factories/projects.rb: project_with_repo factory now calls track_project_repository to ensure a ProjectRepository record exists.
  • ee/spec/factories/geo/project_repository_registry.rb: factory updated to handle both V1 and V2 cases.
  • Specs for Geo::ProjectRepositoryRegistry, GeoNodeStatus, the data management API, and Geo::Secondary::RegistryConsistencyWorker are updated to cover both V1 and V2 paths.

Known hurdle: N+1 queries from the after_find callback

The main technical blocker is an N+1 query problem introduced by the after_find :populate_missing_foreign_keys callback in Geo::ProjectRepositoryRegistry:

# ee/app/models/geo/project_repository_registry.rb
after_find :populate_missing_foreign_keys

def populate_missing_foreign_keys
  return unless has_attribute?(:project_id) && has_attribute?(:project_repository_id)
  return if project.present? && project_repository.present?

  self.project_repository ||= project&.project_repository
  self.project ||= project_repository&.project
end

This callback fires for every record loaded from the database. When loading a batch of registry records (e.g. in cron workers), each record may trigger one or two additional SQL queries to load the project or project_repository association. On large instances with millions of project repositories, this produces a severe N+1 that can degrade performance significantly.

The usual mitigation of adding a scope with includes is not applicable here because geo_project_repository_registry lives in the geo database, while projects and project_repositories live in the main database. Cross-database eager loading is not supported.

Possible approaches:

  • Move the population logic out of after_find entirely, and instead resolve the missing FK lazily on demand (e.g. only when model_accessor is called), accepting that some records may temporarily have only one FK populated.
  • Evaluate whether both FKs need to be populated at all times, or whether the registry can operate correctly with only one FK set and resolve the other only when strictly necessary (e.g. in populate_missing_foreign_keys called explicitly from service-layer code, not from a callback).
  • Consider whether the after_find callback can be removed once the migration period is over and both FKs are always populated by construction.

Acceptance criteria

  • Under geo_project_repository_replication_v2, Geo enumerates project_repositories instead of projects for replication and verification of Git repos.
  • The after_find N+1 in Geo::ProjectRepositoryRegistry is resolved.
  • All existing Geo shared examples pass for both V1 and V2 paths.
Edited by 🤖 GitLab Bot 🤖