Skip to content

Clarify loss of DAG behavior with `needs:pipeline`

Fabio Pitino requested to merge docs/clarify-cross-pipeline-artifacts-with-needs into master

What does this MR do and why?

The reason why I opened this MR is because of a question raised by Support: internal slack link

Clarify that needs:pipeline (as well as other cross-pipeline artifacts downloads) can only be used to download artifacts and does not wait for the needed job to complete.

Example

java_build:
  stage: build
  trigger:
    include:
      - local: java/ci.yml
    strategy: depend
  variables:
    PARENT_PIPELINE_ID: $CI_PIPELINE_ID

docker-build:
  stage: build
  artifacts:
    paths: [images/]
# in java/ci.yml:
java-init:
  needs:
    - pipeline: $PARENT_PIPELINE_ID
      job: docker-build
      artifacts: true

Since docker-build and the trigger job java_build both run in parallel, there is a chance that java-init job attempts to download the artifacts from the parent pipeline that don't exist yet.

When needs:pipeline or needs:project are used (cross-pipeline artifacts download) the needs keyword looses the DAG behavior of waiting for the dependency to complete before running. This occurs because we don't support a cross-pipeline DAG behavior.

For cross-project artifacts download, the user must guarantee that the dependency docker-build completes before the other pipeline job java-init runs.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Fabio Pitino

Merge request reports