Add the ability to search versions in group level for SBoM component versions
This MR adds the ability to fetch versions for a component in group level.
Query plan:
Group level:
SELECT DISTINCT ON ( "sbom_component_versions"."version" ) "sbom_component_versions"."id", "sbom_component_versions"."created_at", "sbom_component_versions"."updated_at", "sbom_component_versions"."component_id", "sbom_component_versions"."version", "sbom_component_versions"."source_package_name", "sbom_component_versions"."organization_id"
from "sbom_component_versions"
WHERE (id IN (WITH RECURSIVE component_versions AS (
SELECT
*
FROM (
SELECT
traversal_ids,
component_name,
component_version_id
FROM
sbom_occurrences
WHERE
traversal_ids >= '{11944369}'
AND traversal_ids < '{11944370}'
AND component_name = 'type-fest' COLLATE "C"
ORDER BY
sbom_occurrences.traversal_ids ASC,
sbom_occurrences.component_name COLLATE "C" ASC,
sbom_occurrences.component_version_id ASC
LIMIT 1
) sub_select
UNION ALL
SELECT
lateral_query.traversal_ids,
lateral_query.component_name,
lateral_query.component_version_id
FROM
component_versions,
LATERAL (
SELECT
sbom_occurrences.traversal_ids,
sbom_occurrences.component_name,
sbom_occurrences.component_version_id
FROM
sbom_occurrences
WHERE
sbom_occurrences.traversal_ids >= '{11944369}'
AND sbom_occurrences.traversal_ids < '{11944370}'
AND (sbom_occurrences.traversal_ids,
sbom_occurrences.component_name,
sbom_occurrences.component_version_id) > (component_versions.traversal_ids,
component_versions.component_name,
component_versions.component_version_id)
AND sbom_occurrences.component_name = 'type-fest' COLLATE "C"
ORDER BY
sbom_occurrences.traversal_ids ASC,
sbom_occurrences.component_name COLLATE "C" ASC,
sbom_occurrences.component_version_id ASC
LIMIT 1
) lateral_query
)
SELECT
component_versions.component_version_id AS id
FROM
component_versions
)) ORDER BY "sbom_component_versions"."version" ASC limit 100
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/38705/commands/118981
https://postgres.ai/console/gitlab/gitlab-production-main/sessions/37580/commands/115088
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37671/commands/115080 (checked by Mehmet)
Project level:
Before:
select
distinct "sbom_component_versions".*
from
sbom_component_versions
inner join sbom_occurrences ON "sbom_occurrences"."component_version_id" = "sbom_component_versions"."id"
where
"sbom_occurrences"."project_id" = 43141468
AND "sbom_occurrences"."component_id" = 1726;
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/36749/commands/112849
After:
Instead of component_id for searching we are using component_name to fix this bug: Project-level dependency list page is incorrect... (#527163 - closed) • Subashis Chakraborty • 18.0
select DISTINCT ON ( "sbom_component_versions"."version" ) "sbom_component_versions"."id", "sbom_component_versions"."created_at", "sbom_component_versions"."updated_at", "sbom_component_versions"."component_id", "sbom_component_versions"."version", "sbom_component_versions"."source_package_name", "sbom_component_versions"."organization_id"
from
sbom_component_versions
inner join sbom_occurrences ON "sbom_occurrences"."component_version_id" = "sbom_component_versions"."id"
where
"sbom_occurrences"."project_id" = 43141468
AND "sbom_occurrences"."component_name" = 'type-fest';
cold: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/38705/commands/118974. warm: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/38705/commands/118972.
Currently there is no index on component_name. If we introduce
exec create index tmp_index on sbom_occurrences using btree (project_id, package_manager, component_name)
by extending the existing index_sbom_occurrences_on_project_id_and_package_manager, it could be faster. But again, this may not be needed. We have similar query !177746 (merged), which is working fine so far without index. May be we should wait to see whether extra index is needed in future as this is in project level.
Steps to setup and validate locally
-
Set up the dependency list on a group
-
Connect to postgres with
gdk psql -
Find a component ID for your project with:
select component_id from sbom_occurrences where component_version_id is not null and project_id = <ID> ; -
Go to
<gdk url>/-/graphql-explorer -
Send this graphql query:
{ group(fullPath: "use your group path") { id componentVersions(componentId: "<ID>") { id version } } }
Example response:
{
"data": {
"group": {
"id": "gid://gitlab/Group/45",
"componentVersions": [
{
"id": "gid://gitlab/Sbom::ComponentVersion/5543",
"version": "6.1.7.2"
}
]
}
},
"correlationId": "01JP0V6MJN986DSNTMXA8TH94B"
}
Related to #521483 (closed) #527163 (closed)