Skip to content

Filter dependencies by project membership

mo khan requested to merge mokhax/438747/explore-visible-dependencies into master

What does this MR do and why?

This change updates the dependency list export for an organization to only include dependencies that are visible to the current user based on project membership. If the current user is an admin then they should be able to view everything. If the current user is an owner of an organization then they should be able to view everything within that organization. All other users are allowed to view dependencies from projects that they have a project authorization to.

#438747 (closed)

Before:

SELECT "sbom_occurrences".*
FROM "sbom_occurrences"
INNER JOIN "projects" ON "sbom_occurrences"."project_id" = "projects"."id"
WHERE "projects"."organization_id" = 1
AND "projects"."archived" = FALSE
ORDER BY "sbom_occurrences"."id" DESC
LIMIT 1000;
Time: 1.453 s
  - planning: 8.329 ms
  - execution: 1.444 s
    - I/O read: 1.394 s
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 4924 (~38.50 MiB) from the buffer pool
  - reads: 1035 (~8.10 MiB) from the OS file cache, including disk I/O
  - dirtied: 564 (~4.40 MiB)
  - writes: 0

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/26934/commands/83867

After:

SELECT *
FROM (
  WITH RECURSIVE "array_cte" AS MATERIALIZED (
    SELECT "project_authorizations"."project_id"
    FROM "project_authorizations"
    WHERE "project_authorizations"."user_id" = 487608
  ),
  "recursive_keyset_cte" AS (
    (
      SELECT NULL::sbom_occurrences AS records, array_cte_project_id_array, sbom_occurrences_id_array, 0::bigint AS count
      FROM (
        SELECT 
          ARRAY_AGG("array_cte"."project_id") AS array_cte_project_id_array,
          ARRAY_AGG("sbom_occurrences"."id") AS sbom_occurrences_id_array
        FROM (
          SELECT "array_cte"."project_id"
          FROM array_cte
        ) array_cte
        LEFT JOIN LATERAL (
          SELECT "sbom_occurrences"."id" AS id
          FROM "sbom_occurrences"
          INNER JOIN "projects" ON "sbom_occurrences"."project_id" = "projects"."id"
          WHERE "projects"."organization_id" = 1
          AND "projects"."archived" = FALSE
          AND "sbom_occurrences"."project_id" = "array_cte"."project_id"
          ORDER BY "sbom_occurrences"."id" ASC
          LIMIT 1
        ) sbom_occurrences ON TRUE
        WHERE "sbom_occurrences"."id" IS NOT NULL
      ) array_scope_lateral_query LIMIT 1
    )
    UNION ALL
    (
      SELECT (
        SELECT sbom_occurrences
        FROM "sbom_occurrences"
        WHERE "sbom_occurrences"."id" = recursive_keyset_cte.sbom_occurrences_id_array[position]
        LIMIT 1
      ),
      array_cte_project_id_array,
      recursive_keyset_cte.sbom_occurrences_id_array[:position_query.position-1]||next_cursor_values.id||recursive_keyset_cte.sbom_occurrences_id_array[position_query.position+1:],
      recursive_keyset_cte.count + 1
      FROM recursive_keyset_cte,
      LATERAL (
        SELECT id, position
        FROM UNNEST(sbom_occurrences_id_array) WITH ORDINALITY AS u(id, position)
        WHERE id IS NOT NULL
        ORDER BY 1 ASC
        LIMIT 1
      ) AS position_query,
      LATERAL (
        SELECT "record"."id"
        FROM (VALUES (NULL)) AS nulls
        LEFT JOIN (
          SELECT "sbom_occurrences"."id" AS id
          FROM "sbom_occurrences"
          INNER JOIN "projects" ON "sbom_occurrences"."project_id" = "projects"."id"
          WHERE "projects"."organization_id" = 1
          AND "projects"."archived" = FALSE
          AND "sbom_occurrences"."project_id" = recursive_keyset_cte.array_cte_project_id_array[position]
          AND ("sbom_occurrences"."id" > recursive_keyset_cte.sbom_occurrences_id_array[position])
          ORDER BY "sbom_occurrences"."id" ASC
          LIMIT 1
        ) record ON TRUE LIMIT 1
      ) AS next_cursor_values
    )
  )
  SELECT (records).* 
  FROM "recursive_keyset_cte" AS "sbom_occurrences"
  WHERE (count <> 0)
) sbom_occurrences
LIMIT 1000
Time: 13.640 s
  - planning: 9.093 ms
  - execution: 13.631 s
    - I/O read: 12.920 s
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 119012 (~929.80 MiB) from the buffer pool
  - reads: 14056 (~109.80 MiB) from the OS file cache, including disk I/O
  - dirtied: 1842 (~14.40 MiB)
  - writes: 0

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27583/commands/85911

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Owner Developer
owner developer

How to set up and validate locally

  1. In rails console enable the experiment fully
    Feature.enable(:explore_dependencies)
  2. Visit http://127.0.0.1:3000/explore/dependencies as a user with the Developer or Maintainer role
  3. Verify the list of dependencies are from projects that the user should have access to
  4. Generate an export and verify the list of dependencies in the downloaded CSV file
Edited by mo khan

Merge request reports