Allow selective sync by orgs for Packages::Nuget::Symbol

What does this MR do and why?

This code change improves how NuGet symbol packages are synchronized across different GitLab servers in a distributed setup (called Geo replication).

The main improvements include:

  1. Better filtering logic: Added a new verifiables method that intelligently determines which symbol files should be verified or replicated based on whether the server is primary or secondary, and what storage options are configured.

  2. Enhanced selective sync: Updated the selective sync feature to work more efficiently by properly filtering records based on project ownership and primary key ranges, rather than loading all records first.

  3. Simplified test structure: Reorganized the test files to be cleaner and more focused, removing some redundant test cases while adding comprehensive tests that cover the new selective sync behavior using shared examples.

  4. Improved replication scope: Added better handling for determining which symbol files should be replicated to secondary servers, taking into account storage location (local vs object storage) and selective sync settings.

The changes make the system more efficient at deciding which NuGet symbol files need to be synchronized between servers, reducing unnecessary data transfer and improving performance in distributed GitLab installations.

References

Related to #585543 (closed)

Database Queries

  • Packages::Nuget::Symbol.replicables_for_current_secondary(1..10000)

    • Raw SQL

      Click to expand
      SELECT
          "packages_nuget_symbols".*
      FROM
          "packages_nuget_symbols"
      WHERE
          "packages_nuget_symbols"."id" BETWEEN 1 AND 10000
          AND "packages_nuget_symbols"."project_id" IN (
              SELECT
                  "projects"."id"
              FROM
                  "projects"
              WHERE
                  "projects"."namespace_id" IN (
                      SELECT
                          "namespaces"."id"
                      FROM
                          "namespaces"
                      WHERE
                          "namespaces"."organization_id" IN (
                              SELECT
                                  "geo_node_organization_links"."organization_id"
                              FROM
                                  "geo_node_organization_links"
                              WHERE
                                  "geo_node_organization_links"."geo_node_id" = 2)));
    • Query Plan: TBD

  • Packages::Nuget::Symbol.pluck_verifiable_ids_in_range(1..10000)

    • Raw SQL

      Click to expand
      SELECT
          "packages_nuget_symbols"."id"
      FROM
          "packages_nuget_symbols"
      WHERE
          "packages_nuget_symbols"."id" BETWEEN 1 AND 10000
          AND "packages_nuget_symbols"."project_id" IN (
              SELECT
                  "projects"."id"
              FROM
                  "projects"
              WHERE
                  "projects"."namespace_id" IN (
                      SELECT
                          "namespaces"."id"
                      FROM
                          "namespaces"
                      WHERE
                          "namespaces"."organization_id" IN (
                              SELECT
                                  "geo_node_organization_links"."organization_id"
                              FROM
                                  "geo_node_organization_links"
                              WHERE
                                  "geo_node_organization_links"."geo_node_id" = 2)));
    • Query Plan: TBD

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports

Loading
Loading