Resolve spec:include locations in the declaring file context

What does this MR do and why?

A spec:include location resolves against the context of the file that added the include: keyword, not the file that declares spec. So a local: location in a file in project A, included by project B, is looked up in project B.

This MR resolves spec:include locations in the context of the file that declares spec. It is behind the feature flag ci_spec_include_own_context, default off, in lib/gitlab/ci/config/external/file/base.rb.

Ci::PipelineCreation::FindPipelineInputsService has the same problem, so the run pipeline input form disagrees with the pipeline when the CI config file lives in another project. Fixed here too, behind the same flag.

This revision also adds log-only instrumentation, so usage can be measured before the flag is enabled.

An earlier attempt blocked spec:include inside included files in !224850 (merged) and was reverted in !226238 (merged) after it broke customer pipelines. This MR blocks nothing, so affected workflows keep working. !250920 (closed) documents the current behavior in the meantime, and gets updated if this fix ships.

Rollout plan

The flag is a beta flag, not gitlab_com_derisk. The measurement period plus holding GitLab Self-Managed and GitLab Dedicated back is longer than the two month lifespan of a derisk flag, and an on-premise install needs a way to turn the change off.

1. Merge with the flag off. The new log line fires only when the file's own context (project or sha) differs from the caller's context and its spec:include contains a local: location. That set is the population the flag would change, so log volume is the blast radius. Query on event: ci_spec_include_diverging_context. The payload also has the caller project_id, the location, the include_type, and the declaring_project_id. Files with no spec: in their content are rejected by a substring check, so they never reach the YAML parse.
2. Query the logs over a couple of weeks. If usage is low or absent, enable the flag.
3. Enable for GitLab.com first. Keep it off for GitLab Self-Managed and GitLab Dedicated until the next major version. That gives anyone relying on the old behavior a clear upgrade boundary instead of a mid-cycle surprise.

Behavior

Given a file in my-group/pipelines declaring spec: include: [local: shared-inputs.yml]:

Consumer	Before	After
The template project running its own pipeline	Works	Works
A consumer project without shared-inputs.yml	Fails with Local file 'shared-inputs.yml' does not exist!	Works, and reads the template project's file
A consumer project with its own shared-inputs.yml	The consumer's value is used at the first include level and the template's value deeper in the chain, so one input name resolves to two values in a single pipeline	The template value is used throughout

Behavior changes when the flag is on

1. A consumer project can currently override an upstream template's inputs by placing a file at the path the upstream uses. That stops working. Consumers pass values through include:inputs with $[[ inputs.x ]], or reference the file with spec:include: project:.
2. A local location inside a file fetched with include:remote or include:template currently reads from the project running the pipeline. It now reports that there is no repository to resolve against. This matches include: local: in a remote file.

Both are covered by specs, in the enabled and disabled flag states.

How to set up and validate locally

1. Create project my-group/pipelines with shared-inputs.yml containing an input image_tag defaulting to "upstream-default", and entrypoint.yml declaring spec: include: [local: shared-inputs.yml] with a deploy job running echo "deploying $[[ inputs.image_tag ]]".
2. Create a second project whose .gitlab-ci.yml only has include: - project: my-group/pipelines, ref: main, file: entrypoint.yml.
3. Run a pipeline on the second project. It fails with Local file 'shared-inputs.yml' does not exist!, and a log line is emitted.
4. Enable the flag with Feature.enable(:ci_spec_include_own_context) and run again. It succeeds and deploy echoes upstream-default.

References

• #590532
• #606741 (closed)
• Rollout issue: #624232