Skip to content

Improve group issue list query performance

Heinrich Lee Yu requested to merge improve-group-issue-list-query into master

What does this MR do and why?

Omits certain query conditions when querying the group issue list as a group member.

This improves the performance of the query.

Before

SELECT "issues".*
FROM "issues"
  INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
  LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE "issues"."author_id" NOT IN (SELECT "banned_users"."user_id" FROM "banned_users")
  AND ( issues.confidential IS NOT TRUE OR (issues.confidential = TRUE AND (issues.author_id = 2880930 OR EXISTS (SELECT TRUE FROM issue_assignees WHERE user_id = 2880930 AND issue_id = issues.id) OR EXISTS (SELECT 1 FROM "project_authorizations" WHERE "project_authorizations"."user_id" = 2880930 AND (project_authorizations.project_id = issues.project_id) AND (project_authorizations.access_level >= 20)))))
  AND "projects"."namespace_id" IN (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND (traversal_ids @> ('{9970}')))
  AND (EXISTS (SELECT 1 FROM "project_authorizations" WHERE "project_authorizations"."user_id" = 2880930 AND (project_authorizations.project_id = projects.id) AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10,20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20,30) OR ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1 FROM "project_authorizations" WHERE "project_authorizations"."user_id" = 2880930 AND (project_authorizations.project_id = project_features.project_id) AND (project_authorizations.access_level >= 10))))
  AND ("issues"."state_id" IN (1))
  AND "projects"."archived" = FALSE
  AND "issues"."issue_type" IN (0, 1, 2)
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC LIMIT 20
Time: 568.023 ms  
  - planning: 3.154 ms  
  - execution: 564.869 ms  
    - I/O read: 0.000 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 359533 (~2.70 GiB) from the buffer pool  
  - reads: 0 from the OS file cache, including disk I/O  
  - dirtied: 0  
  - writes: 0  

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/10942/commands/39307

After

SELECT "issues".*
FROM "issues"
  INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
  LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE "issues"."author_id" NOT IN (SELECT "banned_users"."user_id" FROM "banned_users")
  AND "projects"."namespace_id" IN (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND (traversal_ids @> ('{9970}')))
  AND ("project_features"."issues_access_level" > 0 OR "project_features"."issues_access_level" IS NULL)
  AND ("issues"."state_id" IN (1))
  AND "projects"."archived" = FALSE
  AND "issues"."issue_type" IN (0, 1, 2)
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC LIMIT 20
Time: 481.190 ms  
  - planning: 3.128 ms  
  - execution: 478.062 ms  
    - I/O read: 0.000 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 328083 (~2.50 GiB) from the buffer pool  
  - reads: 0 from the OS file cache, including disk I/O  
  - dirtied: 0  
  - writes: 0  

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/10942/commands/39311

How to set up and validate locally

  1. Open the group issue list as a group member
  2. Check the performance bar for the queries. It's the getIssuesEE GraphQL request.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Heinrich Lee Yu

Merge request reports