Skip to content

Fix project counts on Group Dependency List

mo khan requested to merge mokhax/424211/projects-counts into master

What does this MR do and why?

This change fixes a bug that appears on the Group level Dependency list that was introduced in !132281 (99ebec91). A discussion of this bug can be found in !132281 (comment 1601168757).

In some cases the same dependency might be detected multiple times in the same project. Because of this the occurrence count and project count should differ.

#424211 (closed)

Before:

WITH "our_occurrences" AS MATERIALIZED (
  SELECT DISTINCT ON (sbom_occurrences.component_version_id) sbom_occurrences.*,
    COUNT(sbom_occurrences.id) OVER (PARTITION BY sbom_occurrences.component_version_id) AS occurrence_count,
    COUNT(sbom_occurrences.project_id) OVER (PARTITION BY sbom_occurrences.component_version_id, sbom_occurrences.project_id) AS project_count
  FROM "sbom_occurrences"
  WHERE "sbom_occurrences"."project_id" IN (
    SELECT "projects"."id"
    FROM "projects"
    WHERE "projects"."namespace_id" IN (
      SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
      FROM "namespaces"
      WHERE "namespaces"."type" = 'Group'
      AND (traversal_ids @> ('{9970}'))
    )
    AND "projects"."marked_for_deletion_at" IS NULL
    AND "projects"."pending_delete" = FALSE
  ) 
  AND "sbom_occurrences"."component_version_id" IS NOT NULL
  ORDER BY
    "sbom_occurrences"."component_version_id" DESC,
    "sbom_occurrences"."id" DESC
)
SELECT "sbom_occurrences".*
FROM "our_occurrences" AS "sbom_occurrences"  
ORDER BY "id" ASC
Time: 566.733 ms
  - planning: 10.215 ms
  - execution: 556.518 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 61835 (~483.10 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/23149/commands/74501

After:

WITH "our_occurrences" AS MATERIALIZED (
  SELECT DISTINCT ON (sbom_occurrences.component_version_id) sbom_occurrences.*,
  COUNT(sbom_occurrences.id) OVER (PARTITION BY sbom_occurrences.component_version_id) AS occurrence_count,
  DENSE_RANK() OVER (PARTITION BY sbom_occurrences.component_version_id ORDER BY project_id) AS project_count
  FROM "sbom_occurrences"
  WHERE "sbom_occurrences"."project_id" IN (
    SELECT "projects"."id"
    FROM "projects"
    WHERE "projects"."namespace_id" IN (
      SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, 1)] AS id
      FROM "namespaces"
      WHERE "namespaces"."type" = 'Group'
      AND (traversal_ids @> ('{9970}'))
    )
    AND "projects"."marked_for_deletion_at" IS NULL
    AND "projects"."pending_delete" = FALSE
  )
  AND "sbom_occurrences"."component_version_id" IS NOT NULL
  ORDER BY
    "sbom_occurrences"."component_version_id" DESC,
    "project_count" DESC,
    "sbom_occurrences"."id" DESC
)
SELECT "sbom_occurrences".*
FROM "our_occurrences" AS "sbom_occurrences"
ORDER BY "id" ASC
LIMIT 20 OFFSET 0;
Time: 595.138 ms
  - planning: 10.474 ms
  - execution: 584.664 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 61893 (~483.50 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/23184/commands/74635

Screenshots or screen recordings

Before After
image image image

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by mo khan

Merge request reports