Bundle-backed file source for component include resolution

A pipeline on a Cell resolves a bundled component, and every file it includes, from the bundle instead of the source repository, so the component evaluates with the consuming pipeline's own variables, inputs, and rules.

Why

How a Cell resolves a bundled component:

  1. include: component: gitlab.com/components/opentofu/plan@4.9.0 reaches InstancePath#fetch_content!, and the bundle path from #628309 finds a bundled row.
  2. The Reader downloads that component's bundle file from object storage, once.
  3. The downloaded files and metadata are passed down the include chain on Context. Nested local: includes read them from memory, with no further object-storage calls.

Collection (#630810 (closed)) guarantees that every file the chain can reach is in the bundle

Context is the only object passed from a file to its nested files and to include:rules evaluation, so the bundle goes there

What

Include resolution inside a component reads files from either a source project or a bundle. Add an optional bundle to Context, propagated through mutate the same way as allowed_include_types, and passed in by File::Component when the component resolved from one. Gitlab::Ci::Catalog::Bundle::Reader returns the bundle file's collected files and metadata, and the Ci::Components::FetchService payload carries it in place of project and sha. With no bundle, the existing repository code runs unchanged.

File::Component builds component_data from the metadata: name, version and sha come from it, and reference comes from the consumer's include address. $[[ component.* ]] then renders the same as on the source path. Check the metadata's format version, and fail with a clear error on a version this code does not know.

Things that fail silently if missed:

  • File::Component#component_attrs reads version and reference from the payload. If the bundle path does not fill them from the metadata and the address, $[[ component.version ]] and $[[ component.reference ]] render wrong on a Cell. components/ai-catalog uses both.
  • File::Base#hash is built from project and sha, which are nil in every bundle. Use the bundle's identity instead, or files from different bundles collide.
  • include:rules:exists: inside a bundle always names a project: (collection rejects the other forms). Clause::Exists#change_context builds a fresh Context for it, which is what makes it read the consumer's repository instead of the bundle. Do not propagate the bundle into that context.

File::Local and File::Component read project and sha outside the file fetch too. Each needs a bundle branch:

  • File::Local#validate_context! rejects a context with no project, so every local: include in a bundle fails with "does not have project!".
  • File::Local#metadata builds blob and raw with project_blob_url and project_raw_url from context.project and context.sha. The source repository is not on the Cell, so return nil for both, as File::Component already does for raw.
  • File::Local#log_missing_local_file reads context.project.id, which raises when a path is missing from the bundle and ci_config_local_file_verbose_logging is on.
  • File::Component#masked_blob calls component_payload.fetch(:project) and fetch(:sha), which raise KeyError once the payload carries the bundle in their place.

File::Local#expand_context_attrs needs no change: it passes project and sha (nil here) explicitly, and the bundle reaches nested files through mutate.

Acceptance criteria

  • A bundled component with include: rules: if: on a consumer variable includes or skips the file according to the consumer's own pipeline.
  • include:inputs passed into a bundled component's nested files carry real values.
  • include:rules:exists: with project: $CI_PROJECT_PATH inside a bundled component evaluates against the consumer's repository.
  • $[[ component.name ]], $[[ component.version ]], $[[ component.sha ]] and $[[ component.reference ]] render the same on the bundle path as on the source path for the same include.
  • Include resolution on the default organization is unchanged.

Dependencies

Blocked by #630810 (closed) (bundle format).

Edited by 🤖 GitLab Bot 🤖