Full dotenv variables support in downstream pipelines

What does this MR do and why?

Fix #352828 (closed)

Introduces the ability for bridge jobs to pass down accessible dotenv variables (dependency_variables in the code). This allows downstream pipelines to fully leverage such variables (e.g: as part of rules condition).

Screenshots or screen recordings

For the following pipeline definitions:

# gitlab-ci.yml

stages:
  - create_dotenv
  - child_pipeline

default:
  tags:
    - shared

# Create a .env file
create_dotenv:
  stage: create_dotenv
  script:
    - echo "FOO=bar" > build.env
  artifacts:
    reports:
      dotenv: build.env

# trigger child pipeline
child_pipeline:
  stage: child_pipeline
  trigger:
    include: child_pipeline.yml
    strategy: depend
    forward:
      yaml_variables: true
      pipeline_variables: true

# child_pipeline.yml

stages:
  - test

default:
  tags:
    - shared

test_foo_bar:
  stage: test
  rules:
    - if: '$FOO == "bar"'
  script:
    - echo "Runs when FOO=bar"

test_foo_not_bar:
  stage: test
  rules:
    - if: '$FOO != "bar"'
  script:
    - echo "Runs when FOO!=bar"

The resulting pipeline:

Content of the test_foo_bar job:

How to set up and validate locally

1. Open a new GitLab instance running from this branch
2. Define a child pipeline as part of the CI of a new or existing repo
3. Confirming the expected behavior regarding dotenv variables

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.