Skip to content

Improve performance of group templates finder

Thong Kuah requested to merge performance_projects_new into master

What does this MR do?

Which in turns improves the query to count number of group templates in the New Project page

Old query:

SELECT COUNT(*)
FROM "projects"
WHERE "projects"."namespace_id" IN
    (WITH RECURSIVE "base_and_descendants" AS
       (SELECT "namespaces".*
        FROM "namespaces"
        INNER JOIN "members" ON "namespaces"."id" = "members"."source_id"
        WHERE "members"."type" IN ('GroupMember')
          AND "members"."source_type" = 'Namespace'
          AND "namespaces"."type" IN ('Group')
          AND "members"."user_id" = 2535118
          AND "members"."requested_at" IS NULL
          AND (members.access_level >= 30)
        UNION SELECT "namespaces".*
        FROM "namespaces",
             "base_and_descendants"
        WHERE "namespaces"."type" IN ('Group')
          AND "namespaces"."parent_id" = "base_and_descendants"."id") SELECT DISTINCT "namespaces"."id"
     FROM "base_and_descendants" AS "namespaces"
     WHERE "namespaces"."id" IN
         (WITH RECURSIVE "base_and_descendants" AS
            (SELECT "namespaces".*
             FROM "namespaces"
             WHERE "namespaces"."type" IN ('Group')
               AND (EXISTS
                      (SELECT 1
                       FROM "plans"
                       INNER JOIN "gitlab_subscriptions" ON "gitlab_subscriptions"."hosted_plan_id" = "plans"."id"
                       WHERE "plans"."name" IN ('silver',
                                                'gold')
                         AND (gitlab_subscriptions.namespace_id = namespaces.id)))
             UNION SELECT "namespaces".*
             FROM "namespaces",
                  "base_and_descendants"
             WHERE "namespaces"."type" IN ('Group')
               AND "namespaces"."parent_id" = "base_and_descendants"."id") SELECT DISTINCT "custom_project_templates_group_id"
          FROM "base_and_descendants" AS "namespaces"
          INNER JOIN projects ON projects.namespace_id = namespaces.custom_project_templates_group_id))

New query:

SELECT COUNT(*)
FROM "projects"
WHERE "projects"."namespace_id" IN
    (SELECT DISTINCT "namespaces"."id"
     FROM "namespaces"
     WHERE "namespaces"."type" IN ('Group')
       AND "namespaces"."id" IN
         (WITH RECURSIVE "base_and_descendants" AS
            (SELECT "namespaces".*
             FROM "namespaces"
             INNER JOIN "members" ON "namespaces"."id" = "members"."source_id"
             WHERE "members"."type" IN ('GroupMember')
               AND "members"."source_type" = 'Namespace'
               AND "namespaces"."type" IN ('Group')
               AND "members"."user_id" = 2535118
               AND "members"."requested_at" IS NULL
               AND (members.access_level >= 30)
               AND (EXISTS
                      (SELECT 1
                       FROM "plans"
                       INNER JOIN "gitlab_subscriptions" ON "gitlab_subscriptions"."hosted_plan_id" = "plans"."id"
                       WHERE "plans"."name" IN ('silver',
                                                'gold')
                         AND (gitlab_subscriptions.namespace_id = namespaces.id)))
             UNION SELECT "namespaces".*
             FROM "namespaces",
                  "base_and_descendants"
             WHERE "namespaces"."type" IN ('Group')
               AND "namespaces"."parent_id" = "base_and_descendants"."id") SELECT DISTINCT "custom_project_templates_group_id"
          FROM "base_and_descendants" AS "namespaces"
          INNER JOIN projects ON projects.namespace_id = namespaces.custom_project_templates_group_id))

#33188 (closed)

Does this MR meet the acceptance criteria?

Conformity

Performance and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Thong Kuah

Merge request reports