Geo: Replicate Project Uploads (!226569) · Merge requests · GitLab.org / GitLab

What does this MR do and why?

This code change adds support for tracking and replicating project uploads in GitLab's Geo feature, which helps organizations maintain synchronized copies of their GitLab data across multiple locations.

The main additions include:

New database table: Creates a "project_upload_states" table to track the verification status of uploaded files associated with projects, including when they were verified, any retry attempts, and checksums to ensure data integrity.
Database structure: Adds proper indexing and foreign key relationships to efficiently query and maintain data consistency between projects and their uploaded files.
API integration: Updates the Geo Sites API to include new fields that report statistics about project upload synchronization, such as how many uploads have been synced, verified, or failed.
Monitoring support: Adds new Prometheus metrics to monitor the health and progress of project upload replication across Geo sites.
GraphQL support: Extends the GraphQL API to allow querying project upload registry information, enabling administrators to check replication status through the web interface.

This enhancement ensures that when organizations use GitLab's Geo feature to maintain backup sites, the uploaded files within projects (like images, documents, or other attachments) are properly tracked, synchronized, and verified across all locations, improving data reliability and disaster recovery capabilities.

References

Related to #589915 (closed)

How to set up and validate locally

Prerequisites

A working Geo GDK setup with both primary and secondary running. Follow the Geo development setup guide.

1. Run database migrations

rails db:migrate # on the primary
rails db:migrate:geo # on the secondary

2. Enable the feature flags on the primary

# In Rails console on the primary
Feature.enable(:geo_project_upload_replication)
Feature.enable(:geo_project_upload_force_primary_checksumming)

3. Create test data on the primary

Upload a file to a project issue or create a project upload through the UI (e.g., attach an image to an issue description or comment). Alternatively, use the Rails console:

# In Rails console on the primary
project = Project.first
file = CarrierWaveStringFile.new_file(
  file_content: "Seeded upload file in project #{project.full_path}",
  filename: 'seeded_upload.txt',
  content_type: 'text/plain'
)

UploadService.new(project, file, FileUploader).execute

Verify the upload exists in the project_uploads partition:

Geo::ProjectUpload.count
# Should be > 0

4. Verify checksumming on the primary

Wait for the verification worker to process, or trigger it manually:

# In Rails console on the primary
Geo::ProjectUpload.first.replicator.verify
Geo::ProjectUpload.first.project_upload_state.reload
Geo::ProjectUpload.first.project_upload_state.verification_state
# Should be 2 (verification_succeeded)

5. Verify replication on the secondary

Once the upload is created on the primary, Geo will automatically replicate it to the secondary. Check the sync status in the secondary Rails console:

# In Rails console on the secondary
Geo::ProjectUploadRegistry.count
# Should be > 0

registry = Geo::ProjectUploadRegistry.last
registry.state
# Should be 2 (synced)

If the registry is empty or not yet synced, you can manually trigger sync:

# In Rails console on the secondary
Geo::ProjectUploadReplicator.new(model_record_id: Geo::ProjectUpload.first.id).sync

6. Verify verification on the secondary

# In Rails console on the secondary
registry = Geo::ProjectUploadRegistry.last
registry.reload
registry.verification_state
# Should be 2 (verification_succeeded)

7. Test GraphQL API on the secondary

Note: You must be logged in as an admin user. Non-admin users will get null for Geo-related queries.

Note: When querying from the secondary's GraphQL explorer, add a custom header REQUEST_PATH with the value `/api/v4/geo/node_proxy/{node_id}/graphql

Open the GraphQL explorer on the secondary instance (http://<secondary-url>/-/graphql-explorer) and run:

query {
  geoNode {
    name
    primary
    projectUploadRegistries {
      nodes {
        id
        state
        verificationState
        projectUploadId
        lastSyncedAt
        verifiedAt
      }
    }
  }
}

Expected result: you should see registry entries with state: "SYNCED" and verificationState: "VERIFIED".

8. Verify Geo Sites API

Check the Geo Sites API includes the new project upload statistics:

curl --header "PRIVATE-TOKEN: <your-token>" "http://<primary-url>/api/v4/geo_sites/status"

Look for the new fields in the response:

project_uploads_count
project_uploads_checksummed_count
project_uploads_checksum_failed_count
project_uploads_synced_count
project_uploads_failed_count
project_uploads_registry_count
project_uploads_synced_in_percentage
project_uploads_verified_in_percentage

9. Verify Geo admin page

Visit /admin/geo/sites on the secondary and confirm that "Project Uploads" appears as a new data type with replication and verification progress.

Database Queries

Selective Sync Disabled:

Raw SQL

Click to expand

SELECT
    "project_uploads".*
FROM
    "project_uploads"
WHERE
    "project_uploads"."id" BETWEEN 1 AND 10000;

Query Plan: https://explain.depesz.com/s/isyNh

Selective Sync by Groups:

Raw SQL

Click to expand

SELECT
    "project_uploads".*
FROM
    "project_uploads"
WHERE
    "project_uploads"."id" BETWEEN 1 AND 10000
    AND "project_uploads"."project_id" IN (
        SELECT
            "projects"."id"
        FROM
            "projects"
        WHERE
            "projects"."namespace_id" IN ( WITH RECURSIVE "base_and_descendants" AS (
(
                        SELECT
                            "geo_node_namespace_links"."namespace_id" AS id
                        FROM
                            "geo_node_namespace_links"
                        WHERE
                            "geo_node_namespace_links"."geo_node_id" = 2)
                    UNION (
                        SELECT
                            "namespaces"."id"
                        FROM
                            "namespaces",
                            "base_and_descendants"
                        WHERE
                            "namespaces"."parent_id" = "base_and_descendants"."id"))
                    SELECT
                        "id"
                    FROM
                        "base_and_descendants" AS "namespaces"));

Query Plan: https://explain.depesz.com/s/jddz6

Selective Sync by Organizations:

Raw SQL

Click to expand

SELECT
    "project_uploads".*
FROM
    "project_uploads"
WHERE
    "project_uploads"."id" BETWEEN 1 AND 10000
    AND "project_uploads"."project_id" IN (
        SELECT
            "projects"."id"
        FROM
            "projects"
        WHERE
            "projects"."organization_id" IN (
                SELECT
                    "organizations"."id"
                FROM
                    "organizations"
                    INNER JOIN "geo_node_organization_links" ON "organizations"."id" = "geo_node_organization_links"."organization_id"
                WHERE
                    "geo_node_organization_links"."geo_node_id" = 2));

Query Plan: https://explain.depesz.com/s/YPw6

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited Mar 10, 2026 by Douglas Barbosa Alexandre

Geo: Replicate Project Uploads

What does this MR do and why?

References

How to set up and validate locally

Prerequisites

1. Run database migrations

2. Enable the feature flags on the primary

3. Create test data on the primary

4. Verify checksumming on the primary

5. Verify replication on the secondary

6. Verify verification on the secondary

7. Test GraphQL API on the secondary

8. Verify Geo Sites API

9. Verify Geo admin page

Database Queries

MR acceptance checklist

Merge request reports