Add the ability to search versions in group level for SBoM component versions

This MR adds the ability to fetch versions for a component in group level.

Query plan:

Group level:
SELECT DISTINCT ON ( "sbom_component_versions"."version" ) "sbom_component_versions"."id", "sbom_component_versions"."created_at", "sbom_component_versions"."updated_at", "sbom_component_versions"."component_id", "sbom_component_versions"."version", "sbom_component_versions"."source_package_name", "sbom_component_versions"."organization_id" 
from "sbom_component_versions"
WHERE (id IN (WITH RECURSIVE component_versions AS (
  SELECT
    *
  FROM (
      SELECT
        traversal_ids,
        component_name,
        component_version_id
      FROM
        sbom_occurrences
      WHERE
        traversal_ids >= '{11944369}'
        AND traversal_ids < '{11944370}'
        AND component_name = 'type-fest' COLLATE "C"
        ORDER BY
          sbom_occurrences.traversal_ids ASC,
          sbom_occurrences.component_name COLLATE "C" ASC,
          sbom_occurrences.component_version_id ASC
        LIMIT 1
    ) sub_select
  UNION ALL
  SELECT
    lateral_query.traversal_ids,
    lateral_query.component_name,
    lateral_query.component_version_id
  FROM
    component_versions,
    LATERAL (
      SELECT
        sbom_occurrences.traversal_ids,
        sbom_occurrences.component_name,
        sbom_occurrences.component_version_id
      FROM
        sbom_occurrences
      WHERE
        sbom_occurrences.traversal_ids >= '{11944369}'
        AND sbom_occurrences.traversal_ids < '{11944370}'
        AND (sbom_occurrences.traversal_ids,
          sbom_occurrences.component_name,
          sbom_occurrences.component_version_id) > (component_versions.traversal_ids,
          component_versions.component_name,
          component_versions.component_version_id)
        AND sbom_occurrences.component_name = 'type-fest' COLLATE "C"
        ORDER BY
          sbom_occurrences.traversal_ids ASC,
          sbom_occurrences.component_name COLLATE "C" ASC,
          sbom_occurrences.component_version_id ASC
      LIMIT 1
    ) lateral_query
)
SELECT
  component_versions.component_version_id AS id
FROM
  component_versions
)) ORDER BY "sbom_component_versions"."version" ASC limit 100

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/38705/commands/118981

https://postgres.ai/console/gitlab/gitlab-production-main/sessions/37580/commands/115088

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37671/commands/115080 (checked by Mehmet)

Project level:

Before:

select 
  distinct "sbom_component_versions".* 
from 
  sbom_component_versions 
  inner join sbom_occurrences ON "sbom_occurrences"."component_version_id" = "sbom_component_versions"."id" 
where 
  "sbom_occurrences"."project_id" = 43141468 
  AND "sbom_occurrences"."component_id" = 1726;

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/36749/commands/112849

After:

Instead of component_id for searching we are using component_name to fix this bug: Project-level dependency list page is incorrect... (#527163 - closed) • Subashis Chakraborty • 18.0

select DISTINCT ON ( "sbom_component_versions"."version" ) "sbom_component_versions"."id", "sbom_component_versions"."created_at", "sbom_component_versions"."updated_at", "sbom_component_versions"."component_id", "sbom_component_versions"."version", "sbom_component_versions"."source_package_name", "sbom_component_versions"."organization_id"
from 
  sbom_component_versions 
  inner join sbom_occurrences ON "sbom_occurrences"."component_version_id" = "sbom_component_versions"."id" 
where 
  "sbom_occurrences"."project_id" = 43141468 
  AND "sbom_occurrences"."component_name" = 'type-fest';

cold: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/38705/commands/118974. warm: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/38705/commands/118972.

Currently there is no index on component_name. If we introduce

exec create index tmp_index on sbom_occurrences using btree (project_id, package_manager, component_name) 

by extending the existing index_sbom_occurrences_on_project_id_and_package_manager, it could be faster. But again, this may not be needed. We have similar query !177746 (merged), which is working fine so far without index. May be we should wait to see whether extra index is needed in future as this is in project level.

Steps to setup and validate locally

  1. Set up the dependency list on a group

  2. Connect to postgres with gdk psql

  3. Find a component ID for your project with: select component_id from sbom_occurrences where component_version_id is not null and project_id = <ID> ;

  4. Go to <gdk url>/-/graphql-explorer

  5. Send this graphql query:

    {
     group(fullPath: "use your group path") {
        id
        componentVersions(componentId: "<ID>") {
          id
          version
        }
      }
    }

Example response:

{
  "data": {
    "group": {
      "id": "gid://gitlab/Group/45",
      "componentVersions": [
        {
          "id": "gid://gitlab/Sbom::ComponentVersion/5543",
          "version": "6.1.7.2"
        }
      ]
    }
  },
  "correlationId": "01JP0V6MJN986DSNTMXA8TH94B"
}

Related to #521483 (closed) #527163 (closed)

Edited by Subashis Chakraborty

Merge request reports

Loading