Add support for missing parent filtering on work items queries
What does this MR do and why?
This MR adds support for parent wildcard id filtering on work items queries, as well as negated parent id filtering, to bring parity with the legacy issues API. The intention here is to match the behaviour of the existing epic filtering using the work item parent links table, while maintaining the performance of the existing queries.
References
Screenshots or screen recordings
Not provided
How to set up and validate locally
- In an environment with the GDK running, navigate to http://localhost:3000/-/graphql-explorer
- In a project, create a hierarchical relationship between 2 work items by setting the parent. For example by:
- Setting the parent of an issue to an epic
- Setting the parent of a task to an issue
- Create some more work items without assigning a parent
- Write a query to fetch work items in the project, specifying
parentWildcardId: ANY. e.g:
query q {
namespace(fullPath: "flightjs/flight") {
workItems(parentWildcardId: ANY) {
nodes {
title
}
}
}
}
- Observe that only work items assigned a parent are returned in the query response
- Repeat the above, and set
parentWildcardId: NONE. Observe that only work items without a parent are returned in the query - Write a query to fetch work items who do not have a specific parent with the
not: { parent_ids }arg. e.g:
query q {
namespace(fullPath: "flightjs/flight") {
workItems(not: { parentIds: ["gid://gitlab/WorkItem/748"] }) {
nodes {
title
}
}
}
}
- Observe the results include only work items who do not have the specified parent
- Repeat the above queries on a group / group namespace instead of a project. Observe that the filters have the same effect
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Database
Raw SQL
Project Level Queries
Legacy Query `ANY`
SELECT "issues".*
FROM "issues"
INNER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "issues"."project_id" = 278964
AND "issues"."state_id" = 1
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `ANY`
SELECT "issues".*
FROM "issues"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "issues"."project_id" = 278964
AND "issues"."state_id" = 1
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (EXISTS (SELECT 1 FROM "work_item_parent_links" WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `NONE`
SELECT "issues".*
FROM "issues"
LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "issues"."project_id" = 278964
AND "issues"."state_id" = 1
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND "epic_issues"."epic_id" IS NULL
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `NONE`
SELECT "issues".*
FROM "issues"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "issues"."project_id" = 278964
AND "issues"."state_id" = 1
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (NOT EXISTS (SELECT 1
FROM "work_item_parent_links"
WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `!=`
SELECT "issues".*
FROM "issues"
LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "issues"."project_id" = 278964
AND "issues"."state_id" = 1
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (epic_issues.epic_id NOT IN (273703) OR epic_issues.epic_id IS NULL)
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `!=`
SELECT "issues".*
FROM "issues"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "issues"."project_id" = 278964
AND "issues"."state_id" = 1
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (NOT EXISTS (SELECT 1
FROM "work_item_parent_links"
WHERE "work_item_parent_links"."work_item_id" = "issues"."id"
AND "work_item_parent_links"."work_item_parent_id" IN (145898415)))
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
Group Level Queries
Legacy Query `ANY`
SELECT "issues".*
FROM "issues"
INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
INNER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
FROM UNNEST(COALESCE((SELECT ids
FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
FROM "namespace_descendants"
WHERE "namespace_descendants"."outdated_at" IS NULL
AND "namespace_descendants"."namespace_id" = 9970) cached_query),
(SELECT ids
FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
AND (EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = projects.id)
AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = project_features.project_id)
AND (project_authorizations.access_level >= 10))))
AND "issues"."state_id" = 1
AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `ANY`
SELECT "issues".*
FROM "issues"
INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
FROM UNNEST(COALESCE((SELECT ids
FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
FROM "namespace_descendants"
WHERE "namespace_descendants"."outdated_at" IS NULL
AND "namespace_descendants"."namespace_id" = 9970) cached_query),
(SELECT ids
FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
AND (EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = projects.id)
AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = project_features.project_id)
AND (project_authorizations.access_level >= 10))))
AND "issues"."state_id" = 1
AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (EXISTS (SELECT 1 FROM "work_item_parent_links" WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `NONE`
SELECT "issues".*
FROM "issues"
INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
FROM UNNEST(COALESCE((SELECT ids
FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
FROM "namespace_descendants"
WHERE "namespace_descendants"."outdated_at" IS NULL
AND "namespace_descendants"."namespace_id" = 9970) cached_query),
(SELECT ids
FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
AND (EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = projects.id)
AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = project_features.project_id)
AND (project_authorizations.access_level >= 10))))
AND "issues"."state_id" = 1
AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND "epic_issues"."epic_id" IS NULL
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `NONE`
SELECT "issues".*
FROM "issues"
INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
FROM UNNEST(COALESCE((SELECT ids
FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
FROM "namespace_descendants"
WHERE "namespace_descendants"."outdated_at" IS NULL
AND "namespace_descendants"."namespace_id" = 9970) cached_query),
(SELECT ids
FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
AND (EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = projects.id)
AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = project_features.project_id)
AND (project_authorizations.access_level >= 10))))
AND "issues"."state_id" = 1
AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (NOT EXISTS (SELECT 1
FROM "work_item_parent_links"
WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `!=`
SELECT "issues".*
FROM "issues"
INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
FROM UNNEST(COALESCE((SELECT ids
FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
FROM "namespace_descendants"
WHERE "namespace_descendants"."outdated_at" IS NULL
AND "namespace_descendants"."namespace_id" = 9970) cached_query),
(SELECT ids
FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
AND (EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = projects.id)
AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = project_features.project_id)
AND (project_authorizations.access_level >= 10))))
AND "issues"."state_id" = 1
AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (epic_issues.epic_id NOT IN (273703) OR epic_issues.epic_id IS NULL)
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `!=`
SELECT "issues".*
FROM "issues"
INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
FROM UNNEST(COALESCE((SELECT ids
FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
FROM "namespace_descendants"
WHERE "namespace_descendants"."outdated_at" IS NULL
AND "namespace_descendants"."namespace_id" = 9970) cached_query),
(SELECT ids
FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
AND (EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = projects.id)
AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = 25345642
AND (project_authorizations.project_id = project_features.project_id)
AND (project_authorizations.access_level >= 10))))
AND "issues"."state_id" = 1
AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
AND (NOT EXISTS (SELECT 1
FROM "work_item_parent_links"
WHERE "work_item_parent_links"."work_item_id" = "issues"."id"
AND "work_item_parent_links"."work_item_parent_id" IN (145898415)))
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
Query Plans
- While it is noted that the performance of the group level queries is poor in some cases for both the new and legacy queries, the intention here is to maintain (or improve) performance parity with the existing queries, while we explore options to improve performance of these filters in future (e.g. through
parent_traversal_ids) - For the
!=filter, the frontend will only allow a single parent to be passed (as discussed here). The query plans use a single parent id to reflect this - For the
!=query plan below, we use this epic for the filter
| Filter | Legacy Query | New Query |
|---|---|---|
gitlab-org/gitlab ANY
|
pg.ai | pg.ai |
gitlab-org/gitlab NONE
|
pg.ai | pg.ai |
gitlab-org/gitlab !=
|
pg.ai | pg.ai |
gitlab-org ANY
|
pg.ai | pg.ai |
gitlab-org NONE
|
pg.ai | pg.ai |
gitlab-org !=
|
pg.ai | pg.ai |
Edited by Matt D'Angelo