`parallel:` job templates generate jobs with incorrect names and lead to incorrect `needs:` reference and unusable artifacts
Summary
When defining parallel jobs (ie. using parallel:
), their name in the UI correctly shows identifiable information but the "internal" name (used by the CI) is seemingly copied from the job template instead, discarding the parallel:
information and leading to duplicate names overwriting eachother.
This results in references to it (such as needs:
) always pointing to the last job in the list.
Steps to reproduce
This .gitlab-ci.yml
is accepted by GitLab CI and shows GitLab's buggy behaviour as a result (which can be verified by the fact all but the last job in the test
stage failing):
# Defined here so that we can be sure the two stages define the exact same jobs each.
.parallel-jobs:
parallel:
matrix:
- FOO: [a, b, c]
build:
stage: build
extends: .parallel-jobs
script:
- echo $FOO > foo
artifacts:
paths:
- foo
test:
stage: test
extends: .parallel-jobs
needs:
- build
script:
- '[ "$(cat foo)" = "$FOO" ]'
Changing needs:
to something like build: [$FOO]
is rejected by GitLab CI even though it (or something equivalent) would prevent the bug by identifying the right job.
ci yaml invalid: test: [a] job: undefined need: build: [$FOO]
What is the current bug behavior?
needs: build
is accepted by GitLab and references only one of the build jobs, breaking the pipeline.
What is the expected correct behavior?
needs: build
is rejected as invalid because build
is a template of parallel:
jobs, not a real job, and another scheme is used to identify the correct build
job, such as build[$FOO]
for instance.
Possible fixes
#254821 (closed) requests a feature that could (depending on how it is implemented) fix this bug, or at least allow a workaround. To fully resolve this bug though, invalid uses like needs: build
need to be rejected by the CI, lest we confuse and frustrate Sasha, Devon & Rachel.
An alternative would be to somehow make needs: build
detect that build
is a parallel job template, which variable it used, detect that test
has the same variables set, and use their values to identify the correct build job.
I dislike this solution though, because it hides the logic from the user and is likely to result in hard-to-debug issues (eg. test
uses a different variable name but with the same value).
It also sounds a lot more complicated to implement.
~bug