Ability to render a processed (merged) gitlab-ci.yml file via API

Problem to solve

Writing and debugging nested or complex pipelines is not a trivial task. Especially when working around limitations (or bugs) of advanced features (like include), the resulting pipeline could come handy for finding errors quicker.

Intended users

Developer

Further details

Proposal

Build on #30066 (closed)

1. YamlProcessor to add to the Result object the merged YAML (result of Ci::Config#expand_config)
2. in the CI Lint API: add a key merged_config to expose the merged YAML if API request specifies the param include_merged_yaml=true

Possibly we also solve #37915 with this change.

Limitations

There are three "levels" of processing that are possible:

1. each include keyword is processed and the result merged with the main yaml
2. each job using .extends is merged with the given extension
3. the full data structure is being composed and validated recursively. If syntax/logical errors are found it returns the same data as (2). If it succeeds it will include a normalized list of jobs containing also all sensible defaults that GitLab applies. E.g. default allow_failure: false or default: specs copied in each job.

This feature will only go to level 1. Additional layers are possible in follow on issues, if needed.

Permissions and Security

Security Risks:

• If the merged .gitlab-ci.yml contains variables from this project or one of the included project they could leak. The pipeline should be rendered with the permissions of the user who is running the linter, and not giving them access to anything they wouldn't already have access to.

Documentation

Testing

What does success look like, and how can we measure that?

Links / references