Skip to content

JOB-TOKEN is not accepted for single file artifact downloads

Summary

I want to download a specific file from an artifact's ZIP in another pipeline. I tried different methods as described in and I found an issue.

  • downloading artifact ZIP files is possible with JOB-TOKEN
  • downloading single files from within an artifact is only possible with PRIVATE-TOKEN

According to documentation and other API endpoints, I see no reason, why I can download the whole artifact with a JOB-TOKEN but not parts of it.

Steps to reproduce

  1. I created a repository and appropriate pipeline with a job, that stores an artifact with e.g. 3 files inside.

    AllInOne:
  script:
    - echo "file 1" > file1.txt
    - echo "file 2" > file2.txt
    - echo "file 3" > file3.txt
  artifacts:
    paths:
      - file*.txt

  2. I created a new repository with new pipeline and job that should download one of the files in the artifacts.zip

    image: debian:testing-slim

Build:
  variables:
    PROJECT_NAME: Example
    PROJECT_PATH: paebbels%2Fbug-1
#    REF: v0.10.0
    REF: master
    JOB_NAME: AllInOne
  script:
    - apt update && apt install -y curl unzip
#    - "curl --location --header \"JOB-TOKEN: $CI_JOB_TOKEN\" --output $PROJECT_NAME.zip \"https://gitlab.com/api/v4/projects/$PROJECT_PATH/jobs/artifacts/$REF/download?job=$JOB_NAME\""
    - "curl --location --header \"JOB-TOKEN: $CI_JOB_TOKEN\" --output $PROJECT_NAME.txt \"https://gitlab.com/api/v4/projects/$PROJECT_PATH/jobs/artifacts/$REF/raw/text2.txt?job=$JOB_NAME\""
    - cat *.txt

If you use

       - "curl --location --header \"PRIVATE-TOKEN: ******************\" --output $PROJECT_NAME.txt \"https://gitlab.company.de/api/v4/projects/$PROJECT_PATH/jobs/artifacts/$REF/raw/file2.txt?job=$JOB_NAME\""

it doesn't fail.

Results:

  1. CURL call is successful.
  2. HTTP 404 error.
  3. Same call like 2., but with PRIVATE-TOKEN will be successful.

Example Project

The full example is stored in a self hosted GitLab 13.2.2-ee.

I created two repositories:

What is the current bug behavior?

Single file download from within an artifact does not accept JOB-TOKEN to authenticate, but PRIVATE-TOKEN is accepted.

What is the expected correct behavior?

Allow partial single file artifact downloads with JOB-TOKENs.

Relevant logs and/or screenshots

Successful download of whole ZIP file

$ curl --location --header "JOB-TOKEN: $CI_JOB_TOKEN" --output $PROJECT_NAME.zip "https://gitlab.com/api/v4/projects/$PROJECT_PATH/jobs/artifacts/$REF/download?job=$JOB_NAME"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   715  100   715    0     0   4440      0 --:--:-- --:--:-- --:--:--  4440
100   589  100   589    0     0   2778      0 --:--:-- --:--:-- --:--:--  2778
$ unzip -l $PROJECT_NAME.zip
Archive:  Example.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        7  2020-08-05 18:42   file1.txt
        7  2020-08-05 18:42   file2.txt
        7  2020-08-05 18:42   file3.txt
---------                     -------
       21                     3 files
Job succeeded

See job: https://gitlab.com/paebbels/bug-2/-/jobs/672807820#L188

Failing download of single file

$ curl --location --header "JOB-TOKEN: $CI_JOB_TOKEN" --output $PROJECT_NAME.txt "https://gitlab.com/api/v4/projects/$PROJECT_PATH/jobs/artifacts/$REF/raw/text2.txt?job=$JOB_NAME"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    19    0    19    0     0     68      0 --:--:-- --:--:-- --:--:--    68
$ cat *.txt
404 page not found
Job succeeded

See job: https://gitlab.com/paebbels/bug-2/-/jobs/672809773#L188

Output of checks

GitLab Enterprise Edition (Ultimate): 13.2.2
I also tested now on GitLab.com. It's failing here too.

Possible fixes

Check what TOKENs are accepted for the /raw API endpoint compared to the /download API endpoint.

Edited by Patrick Lehmann