Add support for missing parent filtering on work items queries

What does this MR do and why?

This MR adds support for parent wildcard id filtering on work items queries, as well as negated parent id filtering, to bring parity with the legacy issues API. The intention here is to match the behaviour of the existing epic filtering using the work item parent links table, while maintaining the performance of the existing queries.

References

Screenshots or screen recordings

Not provided

How to set up and validate locally

  1. In an environment with the GDK running, navigate to http://localhost:3000/-/graphql-explorer
  2. In a project, create a hierarchical relationship between 2 work items by setting the parent. For example by:
    • Setting the parent of an issue to an epic
    • Setting the parent of a task to an issue
  3. Create some more work items without assigning a parent
  4. Write a query to fetch work items in the project, specifying parentWildcardId: ANY. e.g:
query q {
  namespace(fullPath: "flightjs/flight") {
    workItems(parentWildcardId: ANY) {
      nodes {
        title
      }
    }
  }
}
  1. Observe that only work items assigned a parent are returned in the query response
  2. Repeat the above, and set parentWildcardId: NONE. Observe that only work items without a parent are returned in the query
  3. Write a query to fetch work items who do not have a specific parent with the not: { parent_ids } arg. e.g:
query q {
  namespace(fullPath: "flightjs/flight") {
    workItems(not: { parentIds: ["gid://gitlab/WorkItem/748"] }) {
      nodes {
        title
      }
    }
  }
}
  1. Observe the results include only work items who do not have the specified parent
  2. Repeat the above queries on a group / group namespace instead of a project. Observe that the filters have the same effect

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Database

Raw SQL

Project Level Queries
Legacy Query `ANY`
SELECT "issues".*
FROM "issues"
         INNER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "issues"."project_id" = 278964
  AND "issues"."state_id" = 1
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `ANY`
SELECT "issues".*
FROM "issues"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "issues"."project_id" = 278964
  AND "issues"."state_id" = 1
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (EXISTS (SELECT 1 FROM "work_item_parent_links" WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `NONE`
SELECT "issues".*
FROM "issues"
         LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "issues"."project_id" = 278964
  AND "issues"."state_id" = 1
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND "epic_issues"."epic_id" IS NULL
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `NONE`
SELECT "issues".*
FROM "issues"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "issues"."project_id" = 278964
  AND "issues"."state_id" = 1
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (NOT EXISTS (SELECT 1
                   FROM "work_item_parent_links"
                   WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `!=`
SELECT "issues".*
FROM "issues"
         LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "issues"."project_id" = 278964
  AND "issues"."state_id" = 1
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (epic_issues.epic_id NOT IN (273703) OR epic_issues.epic_id IS NULL)
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `!=`
SELECT "issues".*
FROM "issues"
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "issues"."project_id" = 278964
  AND "issues"."state_id" = 1
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (NOT EXISTS (SELECT 1
                   FROM "work_item_parent_links"
                   WHERE "work_item_parent_links"."work_item_id" = "issues"."id"
                     AND "work_item_parent_links"."work_item_parent_id" IN (145898415)))
ORDER BY "issues"."updated_at" DESC, "issues"."id" DESC
LIMIT 21
Group Level Queries
Legacy Query `ANY`
SELECT "issues".*
FROM "issues"
         INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
         INNER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
         LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
                                    FROM UNNEST(COALESCE((SELECT ids
                                                          FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
                                                                FROM "namespace_descendants"
                                                                WHERE "namespace_descendants"."outdated_at" IS NULL
                                                                  AND "namespace_descendants"."namespace_id" = 9970) cached_query),
                                                         (SELECT ids
                                                          FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
                                                                FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
                                                                      FROM "namespaces"
                                                                      WHERE "namespaces"."type" = 'Group'
                                                                        AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
  AND (EXISTS (SELECT 1
               FROM "project_authorizations"
               WHERE "project_authorizations"."user_id" = 25345642
                 AND (project_authorizations.project_id = projects.id)
                 AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
       ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
                                                                  FROM "project_authorizations"
                                                                  WHERE "project_authorizations"."user_id" = 25345642
                                                                    AND (project_authorizations.project_id = project_features.project_id)
                                                                    AND (project_authorizations.access_level >= 10))))
  AND "issues"."state_id" = 1
  AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `ANY`
SELECT "issues".*
FROM "issues"
         INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
         LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
                                    FROM UNNEST(COALESCE((SELECT ids
                                                          FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
                                                                FROM "namespace_descendants"
                                                                WHERE "namespace_descendants"."outdated_at" IS NULL
                                                                  AND "namespace_descendants"."namespace_id" = 9970) cached_query),
                                                         (SELECT ids
                                                          FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
                                                                FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
                                                                      FROM "namespaces"
                                                                      WHERE "namespaces"."type" = 'Group'
                                                                        AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
  AND (EXISTS (SELECT 1
               FROM "project_authorizations"
               WHERE "project_authorizations"."user_id" = 25345642
                 AND (project_authorizations.project_id = projects.id)
                 AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
       ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
                                                                  FROM "project_authorizations"
                                                                  WHERE "project_authorizations"."user_id" = 25345642
                                                                    AND (project_authorizations.project_id = project_features.project_id)
                                                                    AND (project_authorizations.access_level >= 10))))
  AND "issues"."state_id" = 1
  AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (EXISTS (SELECT 1 FROM "work_item_parent_links" WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `NONE`
SELECT "issues".*
FROM "issues"
         INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
         LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
         LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
                                    FROM UNNEST(COALESCE((SELECT ids
                                                          FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
                                                                FROM "namespace_descendants"
                                                                WHERE "namespace_descendants"."outdated_at" IS NULL
                                                                  AND "namespace_descendants"."namespace_id" = 9970) cached_query),
                                                         (SELECT ids
                                                          FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
                                                                FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
                                                                      FROM "namespaces"
                                                                      WHERE "namespaces"."type" = 'Group'
                                                                        AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
  AND (EXISTS (SELECT 1
               FROM "project_authorizations"
               WHERE "project_authorizations"."user_id" = 25345642
                 AND (project_authorizations.project_id = projects.id)
                 AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
       ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
                                                                  FROM "project_authorizations"
                                                                  WHERE "project_authorizations"."user_id" = 25345642
                                                                    AND (project_authorizations.project_id = project_features.project_id)
                                                                    AND (project_authorizations.access_level >= 10))))
  AND "issues"."state_id" = 1
  AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND "epic_issues"."epic_id" IS NULL
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `NONE`
SELECT "issues".*
FROM "issues"
         INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
         LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
                                    FROM UNNEST(COALESCE((SELECT ids
                                                          FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
                                                                FROM "namespace_descendants"
                                                                WHERE "namespace_descendants"."outdated_at" IS NULL
                                                                  AND "namespace_descendants"."namespace_id" = 9970) cached_query),
                                                         (SELECT ids
                                                          FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
                                                                FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
                                                                      FROM "namespaces"
                                                                      WHERE "namespaces"."type" = 'Group'
                                                                        AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
  AND (EXISTS (SELECT 1
               FROM "project_authorizations"
               WHERE "project_authorizations"."user_id" = 25345642
                 AND (project_authorizations.project_id = projects.id)
                 AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
       ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
                                                                  FROM "project_authorizations"
                                                                  WHERE "project_authorizations"."user_id" = 25345642
                                                                    AND (project_authorizations.project_id = project_features.project_id)
                                                                    AND (project_authorizations.access_level >= 10))))
  AND "issues"."state_id" = 1
  AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (NOT EXISTS (SELECT 1
                   FROM "work_item_parent_links"
                   WHERE "work_item_parent_links"."work_item_id" = "issues"."id"))
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
Legacy Query `!=`
SELECT "issues".*
FROM "issues"
         INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
         LEFT OUTER JOIN "epic_issues" ON "epic_issues"."issue_id" = "issues"."id"
         LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
                                    FROM UNNEST(COALESCE((SELECT ids
                                                          FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
                                                                FROM "namespace_descendants"
                                                                WHERE "namespace_descendants"."outdated_at" IS NULL
                                                                  AND "namespace_descendants"."namespace_id" = 9970) cached_query),
                                                         (SELECT ids
                                                          FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
                                                                FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
                                                                      FROM "namespaces"
                                                                      WHERE "namespaces"."type" = 'Group'
                                                                        AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
  AND (EXISTS (SELECT 1
               FROM "project_authorizations"
               WHERE "project_authorizations"."user_id" = 25345642
                 AND (project_authorizations.project_id = projects.id)
                 AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
       ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
                                                                  FROM "project_authorizations"
                                                                  WHERE "project_authorizations"."user_id" = 25345642
                                                                    AND (project_authorizations.project_id = project_features.project_id)
                                                                    AND (project_authorizations.access_level >= 10))))
  AND "issues"."state_id" = 1
  AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (epic_issues.epic_id NOT IN (273703) OR epic_issues.epic_id IS NULL)
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21
New Query `!=`
SELECT "issues".*
FROM "issues"
         INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
         LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (NOT EXISTS (SELECT 1 FROM "banned_users" WHERE (banned_users.user_id = (issues.author_id + 0))))
  AND "projects"."namespace_id" IN (SELECT "namespaces"."id"
                                    FROM UNNEST(COALESCE((SELECT ids
                                                          FROM (SELECT "namespace_descendants"."self_and_descendant_group_ids" AS ids
                                                                FROM "namespace_descendants"
                                                                WHERE "namespace_descendants"."outdated_at" IS NULL
                                                                  AND "namespace_descendants"."namespace_id" = 9970) cached_query),
                                                         (SELECT ids
                                                          FROM (SELECT ARRAY_AGG("namespaces"."id") AS ids
                                                                FROM (SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
                                                                      FROM "namespaces"
                                                                      WHERE "namespaces"."type" = 'Group'
                                                                        AND (traversal_ids @> ('{9970}'))) namespaces) consistent_query))) AS namespaces(id))
  AND (EXISTS (SELECT 1
               FROM "project_authorizations"
               WHERE "project_authorizations"."user_id" = 25345642
                 AND (project_authorizations.project_id = projects.id)
                 AND (project_authorizations.access_level >= 10)) OR projects.visibility_level IN (10, 20))
  AND ("project_features"."issues_access_level" IS NULL OR "project_features"."issues_access_level" IN (20, 30) OR
       ("project_features"."issues_access_level" = 10 AND EXISTS (SELECT 1
                                                                  FROM "project_authorizations"
                                                                  WHERE "project_authorizations"."user_id" = 25345642
                                                                    AND (project_authorizations.project_id = project_features.project_id)
                                                                    AND (project_authorizations.access_level >= 10))))
  AND "issues"."state_id" = 1
  AND ("issues"."project_id" IS NULL OR "projects"."archived" = FALSE)
  AND "issues"."work_item_type_id" IN (1, 2, 5, 7, 6, 3)
  AND (NOT EXISTS (SELECT 1
                   FROM "work_item_parent_links"
                   WHERE "work_item_parent_links"."work_item_id" = "issues"."id"
                     AND "work_item_parent_links"."work_item_parent_id" IN (145898415)))
ORDER BY "issues"."created_at" DESC, "issues"."id" DESC
LIMIT 21

Query Plans

  • While it is noted that the performance of the group level queries is poor in some cases for both the new and legacy queries, the intention here is to maintain (or improve) performance parity with the existing queries, while we explore options to improve performance of these filters in future (e.g. through parent_traversal_ids)
  • For the != filter, the frontend will only allow a single parent to be passed (as discussed here). The query plans use a single parent id to reflect this
  • For the != query plan below, we use this epic for the filter
Filter Legacy Query New Query
gitlab-org/gitlab ANY pg.ai pg.ai
gitlab-org/gitlab NONE pg.ai pg.ai
gitlab-org/gitlab != pg.ai pg.ai
gitlab-org ANY pg.ai pg.ai
gitlab-org NONE pg.ai pg.ai
gitlab-org != pg.ai pg.ai
Edited by Matt D'Angelo

Merge request reports

Loading