Skip to content

Make registry table as SSOT to verify Projects and Wikis

What does this MR do?

Make Registry tables as SSOT to verify projects and wikis.

Migration output

  • Up

    ❯ be rails geo:db:migrate
    == 20200707011052 AddVerifiedOnPrimaryColumnsToProjectRegistry: migrating =====
    -- add_column(:project_registry, :primary_repository_checksummed, :boolean, {:default=>false, :null=>false})
       -> 0.0023s
    -- add_column(:project_registry, :primary_wiki_checksummed, :boolean, {:default=>false, :null=>false})
       -> 0.0013s
    == 20200707011052 AddVerifiedOnPrimaryColumnsToProjectRegistry: migrated (0.0037s)
  • Down

    ❯ be rails geo:db:rollback
    == 20200707011052 AddVerifiedOnPrimaryColumnsToProjectRegistry: reverting =====
    -- remove_column(:project_registry, :primary_wiki_checksummed, :boolean, {:default=>false, :null=>false})
       -> 0.0025s
    -- remove_column(:project_registry, :primary_repository_checksummed, :boolean, {:default=>false, :null=>false})
       -> 0.0007s
    == 20200707011052 AddVerifiedOnPrimaryColumnsToProjectRegistry: reverted (0.0070s)

Database queries

Follow the plain SQL query generated here the max batch size is 1000:

SELECT
  "project_registry"."project_id"
FROM ((
    SELECT
      "project_registry"."project_id"
    FROM
      "project_registry"
    WHERE
      "project_registry"."repository_verification_checksum_sha" IS NULL
      AND "project_registry"."last_repository_verification_failure" IS NULL
      AND "project_registry"."resync_repository" = FALSE
      AND project_registry.repository_missing_on_primary IS NOT TRUE)
  UNION (
    SELECT
      "project_registry"."project_id"
    FROM
      "project_registry"
    WHERE
      "project_registry"."wiki_verification_checksum_sha" IS NULL
      AND "project_registry"."last_wiki_verification_failure" IS NULL
      AND "project_registry"."resync_wiki" = FALSE
      AND project_registry.wiki_missing_on_primary IS NOT TRUE)) project_registry
LIMIT 1000;

We don't have the primary_*_checksummed columns available on staging to test this query, but the plan without these columns is here: https://explain.depesz.com/s/5pHW.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Related issues

Closes #223250 (closed)

Edited by Mayra Cabrera

Merge request reports