Upstream Pipeline Fails When Child Trigger Pipeline Passes - (Trigger pipeline marked as cancelled)
Summary
We've encountered an issue using Trigger pipelines that impact our deployment process. This issue is seemingly at random, and I have not been able to re-create it, but it causes our upstream project to get stuck in a failed state.
It currently occurs when one of our projects runs a smoke test on another. The trigger pipeline is started; the downstream pipeline passes, but the trigger pipeline is marked as cancelled and the Upstream is marked as failed. Re-trying the pipeline doesn't resolve the state and a new job needs to be created to resolve the issue.
Steps to reproduce
- Create two projects interlinked to one another through a trigger job
- Run tests against the first project that depend on the second to pass
- This Part is unable to be recreated
- Have Downstream pipeline pass
- Have Trigger Pipeline marked as cancelled
- Have Upstream Job marked as failed.
Example Project
Unable to re-create the issue through example project.
What is the current bug behavior?
- Job is started that triggers a downstream pipeline within another project. The job passes for this project, but the Trigger job is marked as Cancelled, and the Upstream is marked as failed
- Retrying the downstream job does not change the status of the Trigger job, and so Upstream is unable to be resolved to pass
What is the expected correct behavior?
- Downstream job -> having passed, should mark trigger job as passes (not cancelled) which would allow upstream job to pass
- Downstream job -> having failed, should mark the trigger job as failed, and upon retrying the downstream job, should update the Trigger job as passed. Upstream Job should reflect status of Trigger job (which should be correctly reported and allowed to update depending on downstream status)
Relevant logs and/or screenshots
On First project side
e2e:
stage: e2e
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: never
trigger:
project: project 2
strategy: depend
On Second Project:
include:
- local: .gitlab/job-definitions/e2e.yml
variables:
CROSS_PROJECT_E2E: 'true'
TEST_RUN_TAG: 'smoke'
stages:
- smoke-test
- e2e-tests
# Smoke test
e2e-smoke:
stage: smoke-test
extends:
- .e2e-smoke-test
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
Runs on: gitlab-runner 13.11.0
Results of GitLab application Check
- Not running self-managed instance and don't have admin access to run checks.
Possible fixes
- Only possible solution is to re-run the whole pipeline. This almost always changes trigger pipeline status from cancelled and updates according to the job passing or failing.