Collect catalog bundles when a version is published

What does this MR do and why?

Publishing a GitLab-maintained catalog version now collects its components into bundles and stores them. This is behind the ci_collect_bundles_on_publish feature flag, default off.

Ci::Catalog::Resources::Versions::CreateService calls Ci::Catalog::BundledResources::CollectAndStoreService in two steps.

  • prepare collects the components and uploads one document per component. It runs before the publish transaction, so no transaction is held open during the uploads.
  • persist writes the bundled rows. It runs inside the publish transaction, after version.save!. The rows commit or roll back with the version, and they use the saved release date.

The version is not published when collection fails. This covers three cases.

  • Collection rejects a component, for example because of a remote: include. The publish error names the component, the file and the include.
  • The project is not public. CollectAndStoreService rejects it, and the publish fails with that error.
  • Collection or upload raises. The exception is tracked and the publish fails.

A :gitlab_maintained version that cannot be collected cannot be used by the bundle architecture, so it is not published.

Collection runs only when the feature flag is on and the catalog resource is :gitlab_maintained.

The service itself, including the prepare and persist split and the public check, shipped in !260148 (merged). This MR adds only the publish wiring.

References

How to set up and validate locally

  1. Enable the flag in a Rails console.

    Feature.enable(:ci_collect_bundles_on_publish)
  2. Publish a version of a GitLab-maintained catalog resource, either through POST /projects/:id/catalog/publish or by creating a release on the project.

  3. Ci::Catalog::BundledResources::Version.count increases, and the component documents are written to object storage.

  4. Add a remote: include to a component and publish again. The publish fails with an error that names the include.

  5. Disable the flag and publish again. The version is created and nothing is collected.

Specs

bundle exec rspec spec/services/ci/catalog/resources/versions/create_service_spec.rb

MR acceptance checklist

Backend only. No schema or database change. Behind a feature flag, default off. Specs cover both flag states, a rejected or non-public version failing the publish, a raise failing the publish, and a rollback when the rows fail to write.

Edited by Laura Montemayor

Merge request reports

Loading
Loading