Kuberenetes - Runners unable to use Docker Engine of Worker
Summary
GitLab Runner unable to talk to docker daemon on Kubernetes Worker.
time="2018-07-02T03:50:47Z" level=error msg="failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: no such file or directory"
Steps to reproduce
This is my environment
- Kubernetes v1.9.5
- GitLab CE 11.0.1
- GitLab Runner 11.0.0 (gitlab/gitlab-runner:alpine-v11.0.0)
I have installed GiLab Runners as shown here which are working fine.
Important Files
This is my basic CI file.
---
image: docker:latest
services:
- docker:dind
stages:
- build
build:
stage: build
services:
- docker:dind
script:
- id
- ls -lrth
- docker build -t my_test_image .
Config File in Runner
gitlab-runner@gitlab-ci-runner-0:~$ cat /home/gitlab-runner/.gitlab-runner/config.toml
concurrent = 2
check_interval = 0
[[runners]]
name = "gitlab-ci-runner-0"
request_concurrency = 2
url = "http://<IP>/ci"
token = "<token>"
executor = "kubernetes"
[runners.cache]
[runners.kubernetes]
host = ""
bearer_token_overwrite_allowed = false
image = ""
namespace = "gitlab"
namespace_overwrite_allowed = ""
privileged = true
pull_policy = "always"
terminationGracePeriodSeconds = 10
poll_interval = 5
poll_timeout = 360
service_account_overwrite_allowed = ""
pod_annotations_overwrite_allowed = ""
[runners.kubernetes.volumes]
gitlab-runner@gitlab-ci-runner-0:~$
What is the current bug behavior?
Running with gitlab-runner 11.0.0 (5396d320)
on gitlab-ci-runner-0 662eaca1
Using Kubernetes namespace: gitlab
Using Kubernetes executor with image docker:latest ...
Waiting for pod gitlab/runner-662eaca1-project-1-concurrent-0r2kt6 to be running, status is Pending
Waiting for pod gitlab/runner-662eaca1-project-1-concurrent-0r2kt6 to be running, status is Pending
Waiting for pod gitlab/runner-662eaca1-project-1-concurrent-0r2kt6 to be running, status is Pending
Running on runner-662eaca1-project-1-concurrent-0r2kt6 via gitlab-ci-runner-0...
Cloning repository...
Cloning into '/<group_name><project_name>'...
warning: redirecting to https://<gitlab_url>:443/<group_name><project_name>.git/
Checking out 2ca8c62f as master...
Skipping Git submodules setup
$ id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
$ ls -lrth
total 12
-rw-rw-rw- 1 root root 436 Jul 2 03:50 requirements.txt
-rw-rw-rw- 1 root root 45 Jul 2 03:50 README.md
-rw-rw-rw- 1 root root 1.3K Jul 2 03:50 Dockerfile
$ docker build -t my_test_image .
time="2018-07-02T03:50:47Z" level=error msg="failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: no such file or directory"
context canceled
ERROR: Job failed: error executing remote command: command terminated with non-zero exit code: Error executing in Docker Container: 1
What is the expected correct behavior?
Runner should be able to reach docker engine running on the worker.
Nasty Workaround
Everything works fine, if I enable Docker API on the Kubernetes worker host and use variable tcp://<worker_ip>:4243
. But this means all gitlab runners will talk to docker engine on just one worker which is not ideal.