Skip to content

Implement Geo support of Nuget Symbols behind a feature flag

Moaz Khalifarequested to merge
574044-implement-geo-support-of-packages-nuget-symbols-behind-a-feature-flag into master

What does this MR do and why?

This MR implements Geo replication and verification support for NuGet symbol files, enabling GitLab Geo secondary sites to replicate and verify NuGet debugging symbols across distributed installations.

Background:

NuGet symbol files (.pdb files) are debugging symbols that developers use to debug NuGet packages. Currently, these files are not replicated to Geo secondary sites, which means:

  • Developers on secondary sites cannot access debugging symbols
  • Symbol files are not verified for data integrity across Geo nodes
  • Selective sync by namespace or shard doesn't work for symbol files

What this MR does:

  1. Adds Geo replication infrastructure for Packages::Nuget::Symbol:
    • Creates Geo::PackagesNugetSymbolRegistry to track replication state
    • Creates Geo::PackagesNugetSymbolState to track verification state
    • Implements Geo::PackagesNugetSymbolReplicator using the blob replication strategy
  2. Enables verification of NuGet symbols:
    • Adds checksum calculation and verification for symbol files
    • Tracks verification state (pending, succeeded, failed)
    • Supports both local and object storage locations
  3. Supports selective sync:
    • Symbols can be selectively synced by namespace (group/project)
    • Symbols can be selectively synced by storage shard
    • Respects object storage sync settings
  4. Feature flags for safe rollout:
    • geo_packages_nuget_symbol_replication - Controls replication functionality
    • geo_packages_nuget_symbol_force_primary_checksumming - Controls verification on primary

Why this is needed:

This brings NuGet symbols to feature parity with other package files in Geo, ensuring developers on secondary sites have the same debugging capabilities as those on primary sites, while maintaining data integrity through verification.

References

Screenshots or screen recordings

N/A

How to set up and validate locally

Prerequisites

  1. Set up a local Geo environment with a primary and secondary node. Follow the Geo development setup guide.

Testing Replication

On the Primary Node:

  1. Enable the feature flags in Rails console:

    Feature.enable(:geo_packages_nuget_symbol_replication)
Feature.enable(:geo_packages_nuget_symbol_force_primary_checksumming)

  2. Create a NuGet symbol file:

    symbol = FactoryBot.create(:nuget_symbol)

  3. Verify the symbol was created and verification started:

    symbol.reload
symbol.packages_nuget_symbol_state
# Should show verification_state as 'pending' or 'verification_succeeded'

On the Secondary Node:

  1. Check that the registry entry was created:

    Geo::PackagesNugetSymbolRegistry.count
# Should be > 0

registry = Geo::PackagesNugetSymbolRegistry.last
registry.packages_nuget_symbol_id
# Should match the symbol ID from primary

  2. Trigger replication:

    Geo::PackagesNugetSymbolReplicator.new(model_record_id: symbol.id).execute

  3. Verify the symbol was replicated:

    registry.reload
registry.state
# Should be 'synced'

registry.verification_state
# Should be 'verification_succeeded'

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #574044 (closed)

Edited by Moaz Khalifa

Merge request reports