"docker push" does not work when running CI job triggered by tag

Versions

  • GitLab EE Version: 11.8.3
  • GitLab Runner Version: 11.8.0

Expected Behavior

Docker Image is successfully pushed to company artifactory with updated CI_COMMIT_TAG

Files

Here is my .gitlab-ci.yml file:

stages:
    - build
    - push

image: docker:stable

#####################################################
############ Variables Used Across Stages ###########
#####################################################
variables:
    DOCKER_DRIVER: overlay2
    ARTIFACTORY: <company-artifactory>
    CONTAINER_IMAGE_BUILT: ${ARTIFACTORY}/stage/${CI_PROJECT_NAME}:${CI_COMMIT_REF_SLUG}_${CI_COMMIT_SHA}
    CONTAINER_IMAGE_VERSION: ${ARTIFACTORY}/prod/${CI_PROJECT_NAME}:${CI_COMMIT_TAG}
    CONTAINER_IMAGE_LATEST: ${ARTIFACTORY}/stage/${CI_PROJECT_NAME}:latest

#####################################################
################## Stage Definitions ################
#####################################################
build_stage:
  stage: build
  tags:
    - marketplace
  script:
    - docker build -t "${CONTAINER_IMAGE_BUILT}" .
  only:
    - master

build_prod:
  stage: build
  tags:
    - marketplace
  script:
    - docker build -t "${CONTAINER_IMAGE_BUILT}" .
  only:
    - tags
  except:
    - branches

push_stage:
  stage: push
  tags:
    - marketplace
  script:
    - docker login -u $CI_ARTIFACT_USER -p $CI_ARTIFACT_PASSWORD $ARTIFACTORY
    - docker tag ${CONTAINER_IMAGE_BUILT} ${CONTAINER_IMAGE_LATEST}
    - docker push ${CONTAINER_IMAGE_LATEST}
  only:
    - master

push_prod:
  stage: push
  tags: 
    - marketplace
  script:
    - docker login -u $CI_ARTIFACT_USER -p $CI_ARTIFACT_PASSWORD $ARTIFACTORY
    - docker tag ${CONTAINER_IMAGE_BUILT} ${CONTAINER_IMAGE_VERSION}
    - echo ${CONTAINER_IMAGE_VERSION}
    - docker push ${CONTAINER_IMAGE_VERSION}
  only:
    - tags
  except:
    - branches

And my GitLab Runner config.toml file:

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "marketrunner"
  url = "<company gitlab ee>"
  token = "<token>"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "alpine:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Problem

When committing to master and triggering push_stage, docker login and docker push work as expected and the docker image is successfully pushed to our company artifactory. With the same credentials, on the same runner, but with a different tag (and even when I tag my code as latest and the CI_COMMIT_TAG == latest) I get the following error:

unauthorized: The client does not have permission to push to the repository.

The GitLab Runner Output can be seen below:

push_stage

push_prod

This error is only encountered when running as a GitLab CI job on the CI runner. If I ssh into the VM that is running the runner and follow the same docker login, docker tag, and docker push commands, it works as expected. This leads me to believe that it is only a problem with GitLab or its runner.

Edited Apr 05, 2019 by Michael Doyle
Assignee Loading
Time tracking Loading