Skip to content

ERROR: Job failed: exit code 132 - error during connect: Get http://docker:2375/v1.40/containers/cakephp-72/json: dial tcp: lookup docker on xxx.xxx.xxx.xxx:53: server misbehaving

Summary

When running a job in gitlab-runner, I'm getting the error:

ERROR: Job failed: exit code 132

I can't exactly see what is causing this issue from the output of the job details; however, I can replicate an error in docker it's self, that I believe might be causing the error. The docker information below.

While I'm showing the below from one of my projects, all of my projects are having the same issue, and all have almost identical .gitlab-ci.yml configs.

Steps to reproduce

  • Setup a projects with a master, and another branch called something like development.
  • Make changes to the development, commit and push changes.
  • Create a merge request with the below .gitlab-ci.yml file below.
  • Have a runner setup with the below /etc/gitlab-runner/config.toml configuration.

The DOCKERFILE for the two images are located here:

.gitlab-ci.yml 
# This is a CakePHP/PHP project that uses Composer

stages:
- test

variables:
  DOCKER_TLS_CERTDIR: ""

cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
    - vendor/
    - .composercache/

.global_settings: &global_settings
    only:
    - merge_requests
    - master
    - tags
    - web

## I believe it's failing before it even gets to this point ##
before_script:
- composer config cache-files-dir .composercache

test:php 7.2:
    stage: test
    image: 302dev/cakephp:7.2-cli
    <<: *global_settings
    script:
    - composer install # Installs dependencies in the ./vendors directory
    - composer ci # Runs the unit tests

test:php 7.3:
    stage: test
    image: 302dev/cakephp:7.3-cli
    <<: *global_settings
    script:
    - composer install
    - composer ci
/etc/gitlab-runner/config.toml 
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "1.ci-runner-2.redacted.com"
  url = "https://gitlab.redacted.com/"
  token = "XX-XXXXXXXXXX-XXXXXX"
  executor = "docker"
  environment = ["GIT_SSL_NO_VERIFY=1", "COMPOSER_MEMORY_LIMIT=-1"]
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    tls_verify = false
    image = "alpine:latest"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/certs/client", "/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0

When I try to run the image directly in docker, it fails. Docker commands:

  • docker run -d --name cakephp-72 302dev/cakephp:7.2-cli
  • docker run -t docker container start -a -i cakephp-72

Actual behavior

The job output from gitlab's frontend: 
[0KRunning with gitlab-runner 13.0.1 (21cb397c)
[0;m[0K  on 1.ci-runner-1.redacted.com ZtJqQYhk
[0;msection_start:1591300176:prepare_executor
[0K[0K[36;1mPreparing the "docker" executor[0;m
[0;m[0KUsing Docker executor with image 302dev/cakephp:7.2-cli ...
[0;m[0KPulling docker image 302dev/cakephp:7.2-cli ...
[0;m[0KUsing docker image sha256:b33c4a554ca840aeedb9b3ca8da7c419ba99751a048aeb3db2114d1514a980d4 for 302dev/cakephp:7.2-cli ...
[0;msection_end:1591300182:prepare_executor
[0Ksection_start:1591300182:prepare_script
[0K[0K[36;1mPreparing environment[0;m
[0;mRunning on runner-ztjqqyhk-project-110-concurrent-0 via runner-1.redacted.com...
section_end:1591300184:prepare_script
[0Ksection_start:1591300184:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m
[0;m[32;1mFetching changes...[0;m
Reinitialized existing Git repository in /builds/sis-plugins/orgs/.git/
From https://gitlab.redacted.com/sis-plugins/orgs
 * [new ref]         refs/pipelines/1460 -> refs/pipelines/1460
   e5ca156..28b009c  1.x-dev             -> origin/1.x-dev
[32;1mChecking out 28b009cd as refs/merge-requests/34/head...[0;m

[32;1mSkipping Git submodules setup[0;m
section_end:1591300186:get_sources
[0Ksection_start:1591300186:restore_cache
[0K[0K[36;1mRestoring cache[0;m
[0;m[32;1mChecking cache for 1-x-dev-7...[0;m
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.[0;m 
[32;1mSuccessfully extracted cache[0;m
section_end:1591300188:restore_cache
[0Ksection_start:1591300188:download_artifacts
[0K[0K[36;1mDownloading artifacts[0;m
[0;msection_end:1591300190:download_artifacts
[0Ksection_start:1591300190:build_script
[0K[0K[36;1mRunning before_script and script[0;m
[0;msection_end:1591300192:build_script
[0Ksection_start:1591300192:after_script
[0K[0K[36;1mRunning after_script[0;m
[0;msection_end:1591300193:after_script
[0Ksection_start:1591300193:upload_artifacts_on_failure
[0K[0K[36;1mUploading artifacts for failed job[0;m
[0;msection_end:1591300195:upload_artifacts_on_failure
[0K[31;1mERROR: Job failed: exit code 132
[0;m
Results when running the above docker commands: 
# docker run -t docker container start -a -i cakephp-72
error during connect: Get http://docker:2375/v1.40/containers/cakephp-72/json: dial tcp: lookup docker on xxx.xxx.xxx.xxx:53: server misbehaving

Expected behavior

  • The job succeeds like it was before the runners were updated.
  • The docker command runs as expected.

Environment description

  • Locally install gitlab-runner and docker on 2 different Vmware virtual machines.
  • Locally installed gitlab-ce/foss on another virtual machine.

Used GitLab Runner, Docker, GitLab version

# gitlab-runner --version
Version:      13.0.1
Git revision: 21cb397c
Git branch:   13-0-stable
GO version:   go1.13.8
Built:        2020-06-01T08:24:47+0000
OS/Arch:      linux/amd64

Docker version:

# docker version
Client: Docker Engine - Community
 Version:           19.03.11
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        42e35e61f3
 Built:             Mon Jun  1 09:13:48 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.11
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       42e35e61f3
  Built:            Mon Jun  1 09:12:26 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

GitLab version

# apt list --installed | grep gitlab
gitlab-ce/bionic,now 13.0.1-ce.0 amd64 [installed,upgradable to: 13.0.4-ce.0]

Possible fixes

I can't seem to find anything that would be causing ERROR: Job failed: exit code 132 error, nor does searching this show anything. Also, I'm not sure how to see the stdout from the process that is throwing that error code. For the http://docker:2375 error, I followed suggestions from the links below, and none of them worked.

I know that I'm running the free/community edition of everything, but any help would be greatly appreciated!

Thank you, Brian French