Address conflicting syntax `needs:pipeline`
In #255983 (comment 444777769) we had a customer requirement to download artifacts from a parent pipeline and not from the latest successful pipeline for a given ref.
To solve this problem we thought of adding a new syntax:
download-artifacts: needs: - job: generate-artifacts pipeline: $CI_PARENT_PIPELINE_ID artifacts: true
This syntax is in line with the principles of cross project artifacts download and artifacts download via
needs within the same pipeline. The difference is that allowing to fetch artifacts from a job in a specific pipeline narrows down the scope and removes potential conflicts with other running pipelines (e.g. same project and same ref).
However, a similar syntax is available today to mirror the status of an upstream pipeline:
upstream-status: needs: - pipeline: some-group/another-project
This syntax turns
upstream-status into a bridge job, opposite of
trigger keyword, and mirrors the status of an upstream pipeline.
Introducing the new syntax side-by-side is possible but it can cause confusion because the differences are too subtle:
# This is considered as a Job that downloads artifacts from a parent pipeline's `build` job rspec: script: rspec needs: - pipeline: $CI_PARENT_PIPELINE_ID job: build artifacts: true # This is considered as a Bridge that mirrors the status of `upstream-project` latest pipeline on `master` upstream-status: needs: - pipeline: some-group/upstream-project
needs keyword is already overloaded of responsibilities:
- allows to define the sequence of jobs (DAG)
- allows to download artifacts from a previous job in the same pipeline
- allows to download artifacts from a job in the same project but different ref
- allows to download artifacts from a job in the same project and same ref
- allows to download artifacts from a job in another project and ref
- allows to mirror the status of an upstream project pipeline
But maybe combining both DAG and artifacts download with
needs is acceptable. However, mirroring the upstream pipeline status could have been done with a different syntax
We could introduce a new syntax for upstream status mirroring that doesn't use
upstream-status: reflect: some-group/upstream-project
This way we could distinguish between bridge and job better. A bridge defines either
reflect, while a job defines
script. Instead the current code allows
needs to be used in Job for dependency management and in Bridge for status mirroring.
Then, we could translate automatically configs in the form
reflect: syntax so it's backward compatible with existing syntax. This allows us to introduce a new syntax while supporting the old one. In %14.0 we could deprecate
needs:pipeline for status mirroring.