Downstream pipeline: attaching trigger to existing dependency pipeline

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Hey GitLab!

I use downstream pipeline and it is a really nice feature. Thanks!

The aims that I try to achieve:

• Use 1 gitlab-runner
• Don't pollute pipelines (to have only one pipeline for specific project version)

For example, we have project_a that depends on project_b:

# variables, etc.

project_b_trigger:
  stage: dependency
  trigger:
    project: "project_b"
    branch: "v1.0.0"
    strategy: depend

# Other project_a build-deploy staff

Let's assume project_b is already built and deployed and we just need to use artifacts of v1.0.0 pipeline. However, current implementation will trigger project_b pipeline again (pipeline pollution problem).

My first suggestion - add configuration option to use existed pipeline (if it exists) and not trigger new pipeline.

project_b_trigger:
  stage: dependency
  trigger:
    project: "project_b"
    branch: "v1.0.0"
    strategy: depend
    policy:   use-existed # use-existed or always

Currently, I resolve this problem in generate_dependency_triggers job where I check the dependency's pipeline exists and if doesn't - generate trigger job. It is workaround.

I would live with this, but this workaround doesn't work properly when dependency pipeline is running, but not finished yet. We cannot wait in loop and check depepdency pipeline is finished since we have only one runner and this loop will occupy resources and project_b will never finish.

so, even we implement the feature from my previous suggestion we will face with pipeline pollution problem again - when a user runs project_b pipeline manually (or it is triggered from another pipeline, e.g. project_c) and then runs project_a pipeline - it will not find project_b finished pipeline (since it is executing) and project_a will create another one pipeline for project_b. Boom! Like that, due to my previous suggestion we have to:

• Check dependency pipeline exists and if yes just use it
• If dependency pipeline is running - attach to it and wait (in downstream meaning - without blocking runner resources).