Read bundled catalog component documents from object storage
What
Adds Gitlab::Ci::Catalog::Bundle::Reader, which reads a compiled component bundle from the cell's own object storage, plus the finders it needs to locate one by natural key: server_fqdn, full_path, semver and component name.
Why
On a Cell, a customer cannot reach the project that publishes a GitLab-maintained CI component, so the component cannot be fetched from its source. Each version is instead compiled into a self-contained bundle and stored on the cell. This is the read side of that work.
There is no caller yet. #600464 (closed) adds one during pipeline creation.
References
- #600464 (closed)
- Epic: &21992
Database review
Three read-only point lookups. Each predicate is fully covered by a unique btree index, so every condition is an index condition rather than a filter.
1. Resolve the resource by natural key
SELECT "catalog_bundled_resources".* FROM "catalog_bundled_resources"
WHERE "server_fqdn" = 'gitlab.com' AND "full_path" = 'gitlab-org/components/opentofu'
ORDER BY "id" ASC LIMIT 1Limit (cost=2.18..2.18 rows=1 width=192)
-> Sort (cost=2.18..2.18 rows=1 width=192)
Sort Key: id
-> Index Scan using index_catalog_bundled_resources_on_server_fqdn_and_full_path on catalog_bundled_resources (cost=0.15..2.17 rows=1 width=192)
Index Cond: ((server_fqdn = 'gitlab.com'::text) AND (full_path = 'gitlab-org/components/opentofu'::text))The Sort comes from .first and runs over at most one row, since the index is unique on both
filtered columns.
2. Resolve the version by semver
SELECT "catalog_bundled_resource_versions".<columns> FROM "catalog_bundled_resource_versions"
WHERE "catalog_bundled_resource_id" = 1 AND "semver_major" = 1 AND "semver_minor" = 2
AND "semver_patch" = 3 AND "semver_prerelease" IS NULL
LIMIT 1Limit (cost=0.15..2.17 rows=1 width=91)
-> Index Scan using idx_catalog_bundled_versions_on_bundled_resource_and_semver on catalog_bundled_resource_versions (cost=0.15..2.17 rows=1 width=91)
Index Cond: ((catalog_bundled_resource_id = 1) AND (semver_major = 1) AND (semver_minor = 2) AND (semver_patch = 3) AND (semver_prerelease IS NULL))The index is NULLS NOT DISTINCT, which is what keeps released versions indexable and unique with a
NULL prerelease.
3. Resolve the component by name within the version
SELECT "catalog_bundled_resource_components".* FROM "catalog_bundled_resource_components"
WHERE "catalog_bundled_version_id" = 1 AND "name" = 'opentofu'
LIMIT 1Limit (cost=0.15..2.17 rows=1 width=130)
-> Index Scan using idx_catalog_bundled_components_on_version_and_name on catalog_bundled_resource_components (cost=0.15..2.17 rows=1 width=130)
Index Cond: ((catalog_bundled_version_id = 1) AND (name = 'opentofu'::text))On the cost numbers. All three tables are empty, locally and in production, because nothing writes them until the publish-time compile in #600454 (closed). So the costs carry no information and database lab would not add any. The index coverage is the part that holds as the tables grow.