Download of upstream project artifact with CI_JOB_TOKEN fails or get's artifact from previous pipeline
Summary
I have a pipeline triggering a downstream project using CI_JOB_TOKEN and waiting for the downstream job to complete. The downstream pipeline uses the api to get an artifact from a job that has already succeded in the upstream pipeline. I'm finding that I'll get a 401 if the upstream has never passed or it will get the artifact from the previous successful run if available.
I would expect the CI_JOB_TOKEN to only be able to access within the pipeline run of a parent. I would also expect it to get the artifact of a job that has passed even if the entire pipeline is complete.
I've had to work around this by querying the upstream project to get the job id of the job with the archive.
Steps to reproduce
Example from the docs fails if the upstream project has not successfully finished any runs.
https://docs.gitlab.com/ee/api/jobs.html#download-the-artifacts-archive
curl --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test"
Workaround:
The upstream project includes project_id, job_name and pipeline_id as trigger vars.
#!/usr/bin/env bash
set -x
set -e
GITLAB_URL=https://gitlab.my-company.example
UPSTREAM_PROJECT_ID=5103
#FAILS: curl -v -k --header "JOB-TOKEN: ${CI_JOB_TOKEN}" -o artifacts_test.zip "${GITLAB_URL}/api/v4/projects/5103/jobs/artifacts/master/download?job=build"
echo ${UPSTREAM_PROJECT_ID}
echo ${UPSTREAM_JOB_NAME}
echo ${UPSTREAM_PIPELINE_ID}
ls -al
curl -v -k --fail \
--header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
-o pipeline_jobs.json \
"${GITLAB_URL}/api/v4/projects/${UPSTREAM_PROJECT_ID}/pipelines/${UPSTREAM_PIPELINE_ID}/jobs?scope=success"
jq . pipeline_jobs.json
UPSTREAM_JOB_ID=$(jq --arg j_name "${UPSTREAM_JOB_NAME}" '.[] | select(.name==$j_name)|.id' pipeline_jobs.json)
# Download the artifacts for the job ID.
curl -v -k --fail \
--header "JOB-TOKEN: ${CI_JOB_TOKEN}" -o artifacts_test.zip \
"${GITLAB_URL}/api/v4/projects/${UPSTREAM_PROJECT_ID}/jobs/${UPSTREAM_JOB_ID}/artifacts"
ls -al
Example Project
What is the current bug behavior?
401 or gets the artifact of a previous successful run.
What is the expected correct behavior?
Get the artifact from the upstream pipeline if the job with the artifact has succeeded but the pipeline is still pending.
Relevant logs and/or screenshots
Output of checks
Tested on 11.4