Docker in Docker Service (DIND) broken on 11.11 runner
Summary
CI job uses docker:dind image as a service for a specific build. Works fine on 11.10 runner, service fails to start on 11.11
Steps to reproduce
.gitlab-ci.yml
build_portable_disposable:
stage: build
services:
- name: docker:dind
alias: dind
variables:
DOCKER_HOST: tcp://dind:2375
script:
- docker/build-portable disposable
--docker-host dind
# NOTE: because our image exists in an isolated dind instance, we need to
# push it if we're going to use it anywhere else
- docker push "$PORTABLE_DISPOSABLE"
Actual behavior
docker:dind
service fails to start
Running with gitlab-runner 11.11.0 (6c154264)
on ci-runner-4 9o5VTDrE
Using Docker executor with image asia.gcr.io/XXXX/ci-docker-image/18-04:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:bed64de70fa1f4d0b5a498791647c45d954cb0306ec2852dbcfb956f4ff3b0d6 for docker:dind ...
Waiting for services to be up and running...
*** WARNING: Service runner-9o5VTDrE-project-110-concurrent-0-docker-0 probably didn't start properly.
Health check error:
service "runner-9o5VTDrE-project-110-concurrent-0-docker-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2019-05-22T21:35:45.319176683Z time="2019-05-22T21:35:45.318940693Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
2019-05-22T21:35:45.319658994Z Failed to load listeners: can't create unix socket /var/run/docker.sock: device or resource busy
*********
Expected behavior
When run on 11.10 CI runner, docker:dind
service starts correctly.
Running with gitlab-runner 11.10.1 (1f513601)
on ci-runner-4 9o5VTDrE
Using Docker executor with image asia.gcr.io/XXXXX/ci-docker-image/18-04:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:bed64de70fa1f4d0b5a498791647c45d954cb0306ec2852dbcfb956f4ff3b0d6 for docker:dind ...
Waiting for services to be up and running...
Pulling docker image asia.gcr.io/XXXXX/ci-docker-image/18-04:latest ...
Using docker image sha256:aaffda7b01e42d225a14e8eceb51d2214dec46613e3f3f93c147991fab497d99 for asia.gcr.io/XXXXX/ci-docker-image/18-04:latest ...
Running on runner-9o5VTDrE-project-110-concurrent-0 via ci-runner-4...
Environment description
Self hosted CE with runners in GCP
$ sudo docker info
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 4
Server Version: 18.09.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-1032-gcp
Operating System: Ubuntu 18.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 14.68GiB
Name: ci-runner-4
ID: P27V:3VWB:PC3J:O5WP:R2S7:BRZ6:ZUHI:WXFX:ZAOW:RLA5:PT5A:S2MX
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
WARNING: No swap limit support
config.toml contents
concurrent = 6
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "ci-runner-4"
request_concurrency = 6
url = "https://version-control.hostname/"
token = "XXXXXX"
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:latest"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
wait_for_services_timeout = 120
shm_size = 0
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
Used GitLab Runner version
$ gitlab-runner --version
Version: 11.10.1
Git revision: 1f513601
Git branch: 11-10-stable
GO version: go1.8.7
Built: 2019-04-24T09:29:18+0000
OS/Arch: linux/amd64
$ gitlab-runner --version
Version: 11.11.0
Git revision: 6c154264
Git branch: 11-11-stable
GO version: go1.8.7
Built: 2019-05-20T20:19:37+0000
OS/Arch: linux/amd64