Extract the CI catalog components included a pipeline

Background context

When a version (aka release) is created for a catalog resource (aka a "published component project" or "CI/CD Catalog project"), a metadata record is created in catalog_resource_components for each valid component file for the given version. We insert the version_id and name for each record to identify the corresponding component file.

All records in catalog_resource_components represent a valid component in the CI/CD Catalog. However, it's possible to have components that are not in the CI/CD catalog, so here we will use these terms to distinguish them:

• catalog component: Refers to a component in the CI/CD catalog (i.e. one that has a corresponding record in catalog_resource_components).
• component: Refers to any component.

What does this MR do and why?

Our overall objective in the parent issue #440382 (closed) is to implement component usage instrumentation for both GitLab.com and Self-Managed instances. As part of this objective, we must determine when a component is "used", which is defined as below:

When a CI/CD Catalog component is included in a pipeline with the include:component keyword.

To track this usage, we will implement a process that's similar to how included_templates are collected and then used later in Ci::Pipeline::Chain::TemplateUsage.

In this MR, we implement the first part of the process: extraction of included_components. The basic idea is that we save the metadata of each component (project, sha, and name) in an array.

In the next step, the resulting YamlProcessor::Result#included_components will be used in a new pipeline creation chain process where we will evoke the find_catalog_component method.

---

Partially resolves Child Task Step 2 of Parent Issue #440382 (closed).

How to set up and validate locally

To test this, we will set up a scenario that involves multiple components to ensure that Ci::YamlProcessor::Result#included_components produces the correct result.

NOTE: In the following steps, replace the <VALUE> placeholders as needed.

1. Create two new projects Project 1 and Project 2 (with README.md files) in a group named Group C.
2. In Project 1, create a new templates folder and add the following files.

component_a.yml

component-a-job:
  script: echo

component_b.yml

component-b-job:
  script: echo

component_c.yml

include:
  - component: gdk.test:3000/group-c/project-1/component_a@2.0.0

component-c-job:
  script: echo

3. In Project 2, create a new templates folder and add the following file:

component_x.yml

component-x-job:
  script: echo

4. For each project:

• Make any change to the README.md file and commit it to branch1; then commit another change to branch2.
• Create tags 1.0.0 and 2.0.0 for each branch respectively.

5. Ensure the following Yaml is valid in the Pipeline editor. This is just to make sure that everything is set up properly.

include:
  - component: gdk.test:3000/group-c/project-1/component_a@1.0.0
  - component: gdk.test:3000/group-c/project-1/component_a@2.0.0
  - component: gdk.test:3000/group-c/project-1/component_b@1.0.0
  - component: gdk.test:3000/group-c/project-1/component_b@2.0.0
  - component: gdk.test:3000/group-c/project-1/component_c@1.0.0
  - component: gdk.test:3000/group-c/project-1/component_c@2.0.0
  - component: gdk.test:3000/group-c/project-2/component_x@1.0.0
  - component: gdk.test:3000/group-c/project-2/component_x@2.0.0

6. Next we will pass in the following pipeline yaml into Gitlab::Ci::YamlProcessor and observe the result.

include:
  - project: group-c/project-1
    file: templates/component_a.yml # Excluded because it's not using `include:component`
    ref: 1.0.0
  - component: gdk.test:3000/group-c/project-1/component_b@main
  - component: gdk.test:3000/group-c/project-1/component_b@2.0.0
  - component: gdk.test:3000/group-c/project-1/component_b@2.0.0 # Duplicate
  - component: gdk.test:3000/group-c/project-1/component_c@1.0.0 # Includes nested component_a@2.0.0
  - component: gdk.test:3000/group-c/project-2/component_x@2.0.0

With the above yaml, we expect included_catalog_components to contain metadata for all unique components except project-1/component_a@1.0.0.

Run the following in the rails console:

project = Project.find(<PROJECT_1_ID>)
current_user = User.find(1) # Root user

yaml = 
  <<~YAML
  include:
    - project: group-c/project-1
      file: templates/component_a.yml
      ref: 1.0.0
    - component: gdk.test:3000/group-c/project-1/component_b@main
    - component: gdk.test:3000/group-c/project-1/component_b@2.0.0
    - component: gdk.test:3000/group-c/project-1/component_b@2.0.0
    - component: gdk.test:3000/group-c/project-1/component_c@1.0.0
    - component: gdk.test:3000/group-c/project-2/component_x@2.0.0
  YAML

result = Gitlab::Ci::YamlProcessor.new(yaml, project: project, user: current_user).execute

result.included_components.map do |c|
  project = c[:component_project]
  sha = c[:component_sha]
  name = c[:component_name]

  "#{project.path}/#{name}@#{project.repository.tag_names_contains(sha, limit: 1).first || 'main'}"
end

And we observe that the output is as expected:

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #440382 (closed)