Download of upstream project artifact with CI_JOB_TOKEN fails or get's artifact from previous pipeline

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

I have a pipeline triggering a downstream project using CI_JOB_TOKEN and waiting for the downstream job to complete. The downstream pipeline uses the api to get an artifact from a job that has already succeded in the upstream pipeline. I'm finding that I'll get a 401 if the upstream has never passed or it will get the artifact from the previous successful run if available.

I would expect the CI_JOB_TOKEN to only be able to access within the pipeline run of a parent. I would also expect it to get the artifact of a job that has passed even if the entire pipeline is complete.

I've had to work around this by querying the upstream project to get the job id of the job with the archive.

Steps to reproduce

Example from the docs fails if the upstream project has not successfully finished any runs.

https://docs.gitlab.com/ee/api/jobs.html#download-the-artifacts-archive

  curl --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test"

Workaround:

The upstream project includes project_id, job_name and pipeline_id as trigger vars.

#!/usr/bin/env bash
set -x
set -e

GITLAB_URL=https://gitlab.my-company.example
UPSTREAM_PROJECT_ID=5103
#FAILS: curl -v -k --header "JOB-TOKEN: ${CI_JOB_TOKEN}" -o artifacts_test.zip "${GITLAB_URL}/api/v4/projects/5103/jobs/artifacts/master/download?job=build"
echo ${UPSTREAM_PROJECT_ID}
echo ${UPSTREAM_JOB_NAME}
echo ${UPSTREAM_PIPELINE_ID}
ls -al 
curl -v -k --fail \
  --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
  -o pipeline_jobs.json \
  "${GITLAB_URL}/api/v4/projects/${UPSTREAM_PROJECT_ID}/pipelines/${UPSTREAM_PIPELINE_ID}/jobs?scope=success" 

jq . pipeline_jobs.json

UPSTREAM_JOB_ID=$(jq --arg j_name "${UPSTREAM_JOB_NAME}" '.[] | select(.name==$j_name)|.id' pipeline_jobs.json)

# Download the artifacts for the job ID.
curl -v -k --fail \
  --header "JOB-TOKEN: ${CI_JOB_TOKEN}" -o artifacts_test.zip \
  "${GITLAB_URL}/api/v4/projects/${UPSTREAM_PROJECT_ID}/jobs/${UPSTREAM_JOB_ID}/artifacts"
ls -al

Example Project

What is the current bug behavior?

401 or gets the artifact of a previous successful run.

What is the expected correct behavior?

Get the artifact from the upstream pipeline if the job with the artifact has succeeded but the pipeline is still pending.

Relevant logs and/or screenshots

Output of checks

Tested on 11.4

Results of GitLab environment info

Results of GitLab application Check

Possible fixes

Edited by 🤖 GitLab Bot 🤖