Skip to content

Remove invalid protected_environment_deploy_access_levels groups

Shinya Maeda requested to merge fix-data-integrity-deploy-access-levels into master

What does this MR do and why?

This MR removes invalid protected_environment_deploy_access_levels records that have a value on "group_id" column, but there are no corresponding records on "namespaces" table. See this thread (Internal Only) for more information.

Currently, this data integrity issue prevents customers from visiting environment page, therefore this is labeled as severity2.

See Environments page does not show Environments (#412260 - closed) for more information.

Estimation on gitlab.com

  • Row Count: 173,720
  • Batch size: 1000 (sub-batch size: 100)
  • Job count: 173 jobs
  • Job interval: 2 min
  • Total Duration: 346 min (5.76 hours)

Query example per batch

SELECT
    "protected_environment_deploy_access_levels"."id"
FROM
    "protected_environment_deploy_access_levels"
    INNER JOIN namespaces ON namespaces.id = protected_environment_deploy_access_levels.group_id
WHERE
    "protected_environment_deploy_access_levels"."id" BETWEEN 1 AND 1000
    AND "protected_environment_deploy_access_levels"."group_id" IS NOT NULL
    AND (namespaces.type = 'User')
ORDER BY
    "protected_environment_deploy_access_levels"."id" ASC
LIMIT
    1

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/18741/commands/62222

SELECT
    "protected_environment_deploy_access_levels"."id"
FROM
    "protected_environment_deploy_access_levels"
    INNER JOIN namespaces ON namespaces.id = protected_environment_deploy_access_levels.group_id
WHERE
    "protected_environment_deploy_access_levels"."id" BETWEEN 1 AND 1000
    AND "protected_environment_deploy_access_levels"."group_id" IS NOT NULL
    AND (namespaces.type = 'User')
    AND "protected_environment_deploy_access_levels"."id" >= 8
ORDER BY
    "protected_environment_deploy_access_levels"."id" ASC
LIMIT
    1 OFFSET 10

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/18741/commands/62223

DELETE FROM
    "protected_environment_deploy_access_levels"
WHERE
    "protected_environment_deploy_access_levels"."id" IN (
        SELECT
            "protected_environment_deploy_access_levels"."id"
        FROM
            "protected_environment_deploy_access_levels"
            INNER JOIN namespaces ON namespaces.id = protected_environment_deploy_access_levels.group_id
        WHERE
            "protected_environment_deploy_access_levels"."id" BETWEEN 1
            AND 1000
            AND "protected_environment_deploy_access_levels"."group_id" IS NOT NULL
            AND (namespaces.type = 'User')
            AND "protected_environment_deploy_access_levels"."id" >= 8
    )

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/18741/commands/62224

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Shinya Maeda

Merge request reports