Skip to content

Use simplified query for user namespace projects

What does this MR do?

Part of #260327 (closed)

While testing the rollout of the feature flag recursive_approach_for_all_projects in #263442 (closed), there was a raised timeout for Namespace#all_projects_with_pages. The generated query performs well for Namespace of type group (= personal namespace; query plan), but dos not perform well for Namespaces with type nil (query plan).

This change will do the simplest fix by not using the recursive lookup for personal namespaces but still keep it for Groups.

Query plans:

For personal namespaces (https://explain.depesz.com/s/SsFi):

EXPLAIN SELECT "projects".*
FROM "projects"
    INNER JOIN "project_pages_metadata" ON "project_pages_metadata"."project_id" = "projects"."id"
WHERE "projects"."namespace_id" = 1 AND "project_pages_metadata"."deployed" = TRUE

Cold cache:

Time: 190.550 ms
  - planning: 1.200 ms
  - execution: 189.350 ms
    - I/O read: 187.158 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 161 (~1.30 MiB) from the buffer pool
  - reads: 93 (~744.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 1 (~8.00 KiB)
  - writes: 0

Warm cache:

Time: 1.550 ms
  - planning: 1.048 ms
  - execution: 0.502 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 251 (~2.00 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

For groups (https://explain.depesz.com/s/VcBS):

EXPLAIN SELECT "projects".*
FROM "projects"
    INNER JOIN "project_pages_metadata" ON "project_pages_metadata"."project_id" = "projects"."id"
WHERE "projects"."namespace_id" IN (WITH RECURSIVE "base_and_descendants" AS 
  (
      (SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 9970)
      UNION
      (
          SELECT "namespaces".*
          FROM "namespaces", "base_and_descendants"
          WHERE "namespaces"."type" = 'Group' AND "namespaces"."parent_id" = "base_and_descendants"."id"
      )
  ) SELECT id FROM "base_and_descendants" AS "namespaces") AND "project_pages_metadata"."deployed" = TRUE

Cold cache:

Time: 698.504 ms
  - planning: 2.181 ms
  - execution: 696.323 ms
    - I/O read: 631.303 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 3810 (~29.80 MiB) from the buffer pool
  - reads: 2240 (~17.50 MiB) from the OS file cache, including disk I/O
  - dirtied: 118 (~944.00 KiB)
  - writes: 0

Warm cache:

Time: 13.390 ms
  - planning: 2.317 ms
  - execution: 11.073 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 6011 (~47.00 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Corinna Gogolok

Merge request reports