Skip to content

Compliance framework filters for namespace projects

What does this MR do and why?

We want to add compliance framework filters on projects API for the following use cases

  1. See all projects using the compliance framework.
  2. See all projects not using a specific compliance framework.
  3. See all projects without a compliance framework.

This MR adds these parameters in the namespace projects resolver.

How to set up and validate locally

  1. Create a group
  2. Create 3 projects, project_a project_b project_c
  3. Create two compliance frameworks for the group. refer https://docs.gitlab.com/ee/user/group/compliance_frameworks.html
  4. Add 1 compliance framework to project_a and another to project_b
  5. Now test the below queries
 group(fullPath: "group_path") {
    name
    projects(complianceFrameworkId: 1) {
      nodes {
        id
        name
      }
      pageInfo {
        endCursor
        startCursor
        hasNextPage
      }
    }
  }

group(fullPath: "group_path") {
    name
    projects(complianceFrameworkIdNot: 1) {
      nodes {
        id
        name
      }
      pageInfo {
        endCursor
        startCursor
        hasNextPage
      }
    }
  }

group(fullPath: "group_path") {
    name
    projects(withoutComplianceFramework: true) {
      nodes {
        id
        name
      }
      pageInfo {
        endCursor
        startCursor
        hasNextPage
      }
    }
  }
  1. for complianceFrameworkId: idOfFirstComplianceFramework, it should return project_a only
  2. for complianceFrameworkIdNot: idOfFirstComplianceFramework it should return both project_b and project_c.
  3. for withoutComplianceFramework: true we should return project_c only.

Query Plans

Query: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/15214/commands/52833

SELECT
    projects.*
FROM
    projects
    INNER JOIN project_compliance_framework_settings compliance_framework_setting ON compliance_framework_setting.project_id = projects.id
WHERE
    projects.namespace_id = 9970
    AND compliance_framework_setting.framework_id = 1

Query Plan:

 Nested Loop Anti Join  (cost=0.73..88.91 rows=22 width=761) (actual time=0.087..1.487 rows=291 loops=1)
   Buffers: shared hit=880
   I/O Timings: read=0.000 write=0.000
   ->  Index Scan using index_projects_on_namespace_id_and_id on public.projects  (cost=0.44..33.51 rows=22 width=761) (actual time=0.041..0.781 rows=291 loops=1)
         Index Cond: (projects.namespace_id = 9970)
         Buffers: shared hit=295
         I/O Timings: read=0.000 write=0.000
   ->  Index Only Scan using index_project_compliance_framework_settings_on_project_id on public.project_compliance_framework_settings  (cost=0.29..2.42 rows=1 width=8) (actual time=0.002..0.002 rows=0 loops=291)
         Index Cond: (project_compliance_framework_settings.project_id = projects.id)
         Heap Fetches: 0
         Buffers: shared hit=585
         I/O Timings: read=0.000 write=0.000

Query: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/15214/commands/52836

SELECT
    projects.*
FROM
    projects
    LEFT OUTER JOIN project_compliance_framework_settings ON project_compliance_framework_settings.project_id = projects.id
WHERE
    projects.namespace_id = 9970
    AND project_compliance_framework_settings.project_id IS NULL

Query Plan:

 Nested Loop Anti Join  (cost=0.73..88.91 rows=22 width=761) (actual time=0.087..1.487 rows=291 loops=1)
   Buffers: shared hit=880
   I/O Timings: read=0.000 write=0.000
   ->  Index Scan using index_projects_on_namespace_id_and_id on public.projects  (cost=0.44..33.51 rows=22 width=761) (actual time=0.041..0.781 rows=291 loops=1)
         Index Cond: (projects.namespace_id = 9970)
         Buffers: shared hit=295
         I/O Timings: read=0.000 write=0.000
   ->  Index Only Scan using index_project_compliance_framework_settings_on_project_id on public.project_compliance_framework_settings  (cost=0.29..2.42 rows=1 width=8) (actual time=0.002..0.002 rows=0 loops=291)
         Index Cond: (project_compliance_framework_settings.project_id = projects.id)
         Heap Fetches: 0
         Buffers: shared hit=585
         I/O Timings: read=0.000 write=0.000

Query: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/15214/commands/52835

SELECT
    projects.*
FROM
    projects
    LEFT OUTER JOIN project_compliance_framework_settings compliance_framework_setting ON compliance_framework_setting.project_id = projects.id
WHERE
    projects.namespace_id = 88
    AND (compliance_framework_setting.framework_id != 6
        OR compliance_framework_setting.framework_id IS NULL)

Query Plan:

 Nested Loop Left Join  (cost=0.73..105.05 rows=22 width=761) (actual time=1.533..3.705 rows=4 loops=1)
   Filter: ((compliance_framework_setting.framework_id <> 6) OR (compliance_framework_setting.framework_id IS NULL))
   Rows Removed by Filter: 0
   Buffers: shared hit=13 read=5
   I/O Timings: read=3.553 write=0.000
   ->  Index Scan using index_projects_on_namespace_id_and_id on public.projects  (cost=0.44..33.51 rows=22 width=761) (actual time=1.476..3.622 rows=4 loops=1)
         Index Cond: (projects.namespace_id = 88)
         Buffers: shared hit=2 read=5
         I/O Timings: read=3.553 write=0.000
   ->  Index Scan using index_project_compliance_framework_settings_on_project_id on public.project_compliance_framework_settings compliance_framework_setting  (cost=0.29..3.24 rows=1 width=16) (actual time=0.016..0.016 rows=0 loops=4)
         Index Cond: (compliance_framework_setting.project_id = projects.id)
         Buffers: shared hit=11
         I/O Timings: read=0.000 write=0.000

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #388188 (closed)

Edited by Harsimar Sandhu

Merge request reports