Skip to content

Errors connecting to Docker socket

Summary

I am receiving errors connecting to the Docker socket during the start of builds, the retries sometimes pass but a good percentage of the time the builds fail. Common error messages are:

  • ERROR: Preparation failed: error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.18/containers/create?name=runner-c23dc1dc-project-35-concurrent-3-build: EOF
  • ERROR: Preparation failed: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
  • ERROR: Job failed (system failure): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

The /var/run/docker.sock exists on the host and Docker is running correctly, I can pull/run images as a normal user that is part of the docker group.

Steps to reproduce

Set up a GitLab runner on Arch Linux

Actual behaviour

Get error messages about not being able to connect to Docker socket or HTTP Docker endpoint

Expected behaviour

A clean build

Relevant logs and/or screenshots

Running with gitlab-ci-multi-runner dev (HEAD)
  on canny (c23dc1dc)
Using Docker executor with image vcatechnology/linux-mint-toolchain:17.3 ...
Using docker image sha256:65009e2043f704d40a314ca7a2f71d056880dddf55f719eba5f5fc3f2b4094bb for predefined container...
Pulling docker image vcatechnology/linux-mint-toolchain:17.3 ...
Using docker image vcatechnology/linux-mint-toolchain:17.3 ID=sha256:978e2de0c80f6af5e0f100f18158e3805d03e1885dc54d8c134ebc8b51827ecf for build container...
ERROR: Preparation failed: error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.18/containers/create?name=runner-c23dc1dc-project-35-concurrent-3-build: EOF
Will be retried in 3s ...
Using Docker executor with image vcatechnology/linux-mint-toolchain:17.3 ...
ERROR: Preparation failed: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Will be retried in 3s ...
Using Docker executor with image vcatechnology/linux-mint-toolchain:17.3 ...
Using docker image sha256:65009e2043f704d40a314ca7a2f71d056880dddf55f719eba5f5fc3f2b4094bb for predefined container...
Pulling docker image vcatechnology/linux-mint-toolchain:17.3 ...
Using docker image vcatechnology/linux-mint-toolchain:17.3 ID=sha256:978e2de0c80f6af5e0f100f18158e3805d03e1885dc54d8c134ebc8b51827ecf for build container...
Running on runner-c23dc1dc-project-35-concurrent-3 via canny...
Cloning repository...
Cloning into '/mnt/builds/tool-chain/tool-chain'...
Checking out 1e171267 as mr-cache-third-party-refactor...
Skipping Git submodules setup

Environment description

Custom installation, using Arch Linux with pacman provided docker and gitlab-runner.

[matt-clarkson@canny ~]$ docker info
Containers: 42
 Running: 1
 Paused: 0
 Stopped: 41
Images: 9
Server Version: 17.04.0-ce
Storage Driver: devicemapper
 Pool Name: docker-thinpool
 Pool Blocksize: 524.3kB
 Base Device Size: 10.74GB
 Backing Filesystem: xfs
 Data file: 
 Metadata file: 
 Data Space Used: 27.7GB
 Data Space Total: 254.5GB
 Data Space Available: 226.8GB
 Metadata Space Used: 6.095MB
 Metadata Space Total: 2.676GB
 Metadata Space Available: 2.67GB
 Thin Pool Minimum Free Space: 25.45GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
 Library Version: 1.02.139 (2017-04-13)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: 
containerd version: 422e31ce907fd9c3833a38d7b8fdd023e5a76e73
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.10.11-1-ARCH
Operating System: Arch Linux
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 47.16GiB
Name: canny
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Used GitLab Runner version

Version:      dev
Git revision: HEAD
Git branch:   HEAD
GO version:   go1.8.1
Built:        Thu, 27 Apr 2017 11:48:06 +0100
OS/Arch:      linux/amd64

This is actually incorrect though - it's using v9.1.0 tag (at the time of submission) as seen in the PKGBUILD]

concurrent = 8
check_interval = 0

[[runners]]
  name = "canny"
  url = "https://gitlab.xyz.com/"
  token = "abcdef0123457890abcdef01234567"
  executor = "docker"
  builds_dir = "/mnt/builds"
  cache_dir = "/mnt/cache"
  [runners.docker]
    tls_verify = false
    image = "vcatechnology/arch-ci"
    privileged = false
    cap_add = ["SYS_PTRACE"]
    disable_cache = false
    extra_hosts = ["prewitt:192.168.4.1", "sobel:192.168.1.250"]
    volumes = [
      "/mnt/gitlab-runner/cache:/mnt/cache:rw",
      "/mnt/gitlab-runner/third-party-prebuilt:/mnt/third-party-prebuilt:rw"
    ]
    shm_size = 0
  [runners.cache]