Skip to content

Issue running build from local image with gitlab-ci-multi-runner >= 1.8.0 (regression)

Summary

Running a build using an image that was created earlier in the build process results in an image not found error. This was functioning in 1.7.1 as I expected, but broke sometime before 1.8.0. I outlined my specific details in https://forum.gitlab.com/t/local-docker-image-not-found/5274.

Steps to reproduce

  • apt-get install gitlab-ci-multi-runner
  • Add a runner from library/docker
export REGISTER_NON_INTERACTIVE=true
export CI_SERVER_URL=https://gitlab.example.com/ci
export RUNNER_EXECUTOR=docker
export RUNNER_NAME=docker-runner-$(hostname)
export REGISTRATION_TOKEN=MY_TOKEN
export RUNNER_TAG_LIST=docker
export DOCKER_IMAGE=docker
export DOCKER_VOLUMES=/var/run/docker.sock:/var/run/docker.sock
gitlab-runner register
  • Create a .gitlab-ci.yaml file that builds an image and then tries to use that image later
build-test:
  stage: prepare
  script:
   - docker build -t test .
  tags:
   - docker
use-test:
  stage: analyze
  script:
    - echo "Hello World
  image: test
  • Attempt to run the build - build fails with library/test image not found error
  • apt-get install gitlab-ci-multi-runner=1.7.1
  • Attempt to run the build - build succeeds

Actual behavior

A docker pull is executed on the image so the build will fail with the following error: ERROR: Build failed: Error: image library/test:latest not found.

Expected behavior

The local image should be used and the build should succeed.

Relevant logs and/or screenshots

These are from my specific case and not the more general case I outlined above:

Error message:

Running with gitlab-ci-multi-runner 1.8.1 (a2efdd4)
Using Docker executor with image tma-test ...
Using locally found image version with exactly the same ID
Pulling docker image tma-test ...
ERROR: Build failed: Error: image library/tma-test:latest not found

.gitlab-ci.yaml

build-test:
  stage: prepare
  script:
   - docker build -t tma-test --build-arg BUNDLE_WITHOUT="" --build-arg RAILS_ENV=development .
  tags:
   - docker
rubocop:
  stage: analyze
  script:
    - bundle exec rubocop
  image: tma-test

Docker runner configuration:

[[runners]]
  name = "docker-runner-build"
  url = "https://gitlab.example.com/ci"
  token = "MY_TOKEN"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "private-repository.com/travel-deploy:latest"
    privileged = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
  [runners.cache]

Docker images on build server

root@build:/etc/gitlab-runner$ docker images
REPOSITORY                                                              TAG                 IMAGE ID            CREATED             SIZE
tma-test                                                                latest

Environment description

  • Custom installation from package
  • Built via user-data launch script on an AWS spot instance. Last build was November 3rd which was working. New build today pulled new
  • Docker executor using the library/docker image mounting /var/run/docker.sock.
  • Docker executor using a custom image created in the previous runner.
  • docker info
root@ip-172-31-10-124:/etc/gitlab-runner# docker info
Containers: 5
 Running: 0
 Paused: 0
 Stopped: 5
Images: 47
Server Version: 1.12.3
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 73
 Dirperm1 Supported: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 3.13.0-91-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.797 GiB
Name: ip-172-31-10-124
ID: FY7H:JI7F:NGB6:OGB5:3MDX:HBW5:OPEU:GSPO:5D7K:ASVB:L7SD:4QC7
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Used GitLab Runner version

1.8.1, but I can also recreate in 1.8.0. When using 1.7.1, this functions as expected.

root@build:/etc/gitlab-runner# gitlab-runner --version
Version:      1.8.1
Git revision: a2efdd4
Git branch:   1-8-stable
GO version:   go1.7.3
Built:        Tue, 29 Nov 2016 13:53:55 +0000
OS/Arch:      linux/amd64