Compile and store bundled catalog versions

What does this MR do and why?

Completes Ci::Catalog::BundledResources::CompileAndStoreService to persist published catalog versions as cell-local bundled records with self-contained component documents in object storage.

Key design decisions:

1. Semver columns copied from source - The version row copies semver_prefixed rather than re-parsing. ObjectKey#dir uses semver.to_s which re-adds v only when prefixed; dropping this would break the read path.

2. Component spec from published source - Spec is copied from the published component, not derived from compiled output (where includes are already resolved).

3. Components saved outside transaction - Each component is saved individually to avoid holding a transaction open across up to 100 uploads. A spec asserts no transaction count increase during saves.

4. Readme stored, not fetched - A consuming cell has no access to the publisher's repository, so the route the source uses (Resources::Version#readme reading from Gitaly) is unavailable there. Storing it is the only way a bundled readme can exist.

Database

Adds a nullable cached_markdown_version column to catalog_bundled_resource_versions. Nothing has ever written this table, so the migration cannot fail on existing rows, and a nullable column with no default is a metadata-only change.

The table already carried readme and readme_html, mirroring catalog_resource_versions, but not the version column CacheMarkdownField needs in order to invalidate rendered HTML.

All writes are single-row upserts or inserts keyed on unique indexes. No bulk statements and no sequential scans.

Known properties

Recompiling a version may not reproduce the same bundle. The compiler inlines local: and component: includes. A version's own files are fixed at its sha, but an included component can be republished. Idempotency holds for the row writes and the object key, not for the bytes stored at that key. Only the backfill can reach this.

Relative links in a stored readme will not resolve on the consuming side, because there is no project context there. Inherent to bundling rather than to this change.

How to validate locally

bundle exec rspec spec/services/ci/catalog/bundled_resources/compile_and_store_service_spec.rb
bundle exec rspec spec/models/ci/catalog/bundled_resources/component_spec.rb
bundle exec rspec spec/models/ci/catalog/bundled_resources/version_spec.rb

Database review

Three write statements and one new scope. All three tables are new and no code writes them yet, so they are empty in production and this merge request is their first writer. Plans from a populated database are not available for that reason.

Each conflict target is served by an existing unique index, all created in db/migrate/:

• index_catalog_bundled_resources_on_server_fqdn_and_full_path
• idx_catalog_bundled_versions_on_bundled_resource_and_semver
• idx_catalog_bundled_components_on_version_and_name

1. Bundled resource upsert, one row

INSERT INTO catalog_bundled_resources
  (server_fqdn, full_path, name, description, latest_released_at, created_at, updated_at)
VALUES ('localhost', 'namespace1/project-1', 'Project-1 Name', NULL, '2018-10-20 18:00:00',
        CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (server_fqdn, full_path)
DO UPDATE SET name = excluded.name,
              description = excluded.description,
              updated_at = excluded.updated_at
RETURNING id

2. Version upsert, one row

INSERT INTO catalog_bundled_resource_versions
  (catalog_bundled_resource_id, released_at, semver_major, semver_minor, semver_patch,
   semver_prerelease, semver_prefixed, created_at, updated_at)
VALUES (55, '2018-10-20 18:00:00', 2, 1, 0, NULL, TRUE, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT (catalog_bundled_resource_id, semver_major, semver_minor, semver_patch, semver_prerelease)
DO UPDATE SET released_at = excluded.released_at,
              semver_prefixed = excluded.semver_prefixed,
              updated_at = CASE WHEN (...unchanged...) THEN updated_at ELSE CURRENT_TIMESTAMP END
RETURNING id

3. Component upsert, one statement for every component of the version

Bounded at 100 rows by Ci::Catalog::ComponentsProject::COMPONENTS_LIMIT.

INSERT INTO catalog_bundled_resource_components
  (name, spec, file, file_store, catalog_bundled_resource_id, catalog_bundled_version_id, created_at)
VALUES ('component', '{"inputs":{"stage":{"default":"test"}}}', 'component.yml', 1, 55, 55, '2026-08-25 15:06:24'),
       ('second', '{}', 'second.yml', 1, 55, 55, '2026-08-25 15:06:24')
ON CONFLICT (catalog_bundled_version_id, name)
DO UPDATE SET spec = excluded.spec,
              file = excluded.file,
              file_store = excluded.file_store
RETURNING id

The update set is restricted with update_only. Without it Rails also writes created_at = excluded.created_at, because this table has no updated_at column, which moved created_at forward every time a version was bundled again. There is a spec for that.

4. New scope: BundledResources::Version.latest

Called as bundled_resource.versions.latest, so it is filtered by catalog_bundled_resource_id, which is the leading column of the unique index above. One row per version of one resource.

SELECT catalog_bundled_resource_versions.*
FROM catalog_bundled_resource_versions
WHERE catalog_bundled_resource_id = 55
  AND semver_major IS NOT NULL
  AND semver_prerelease IS NULL
ORDER BY semver_major DESC, semver_minor DESC, semver_patch DESC,
         CASE WHEN semver_prerelease IS NULL THEN 0 ELSE 1 END,
         REGEXP_REPLACE(semver_prerelease, '[0-9]+', '', 'g') DESC NULLS FIRST,
         COALESCE(NULLIF(REGEXP_REPLACE(semver_prerelease, '[^0-9]', '', 'g'), '')::NUMERIC, 0) DESC
LIMIT 1

Limit
  ->  Sort
        Sort Key: semver_major DESC, semver_minor DESC, semver_patch DESC, ...
        ->  Seq Scan on catalog_bundled_resource_versions
              Filter: (semver_prerelease IS NULL)

The sequential scan is an empty-table artifact. The ordering expressions are not indexable, so the sort is unavoidable; it runs over the versions of a single resource.

References

• Related to #600454 (closed)
• Closes #616490 (closed)