Skip to content

gitlab-runner fails to teardown docker:dind service

Summary

After upgrading to gitlab-runner v17.4.0 each job that uses a docker:dind service fails at the end of the job It seems it's failing to stop the docker:dind

Steps to reproduce

Run the same job with gitlab-runner

  • v17.3.1 (succeeds)
  • v17.4.0 (fails)
.gitlab-ci.yml
docker_build:
  services:
    - docker:dind
  script:
  ...
  - docker build . -f $DOCKERFILE -t $DOCKER_IMAGE_NAME
  - docker tag $DOCKER_IMAGE_NAME $TAG
  ...

Relevant logs and/or screenshots

job log
Saving cache for successful job
03:03
Waiting for pod gitlabcloud/runner-kgyedwqct-project-53701343-concurrent-0-wczv87xq to be running, status is Running
	ContainersNotReady: "containers with unready status: [svc-1]"
	ContainersNotReady: "containers with unready status: [svc-1]"
Waiting for pod gitlabcloud/runner-kgyedwqct-project-53701343-concurrent-0-wczv87xq to be running, status is Running
	ContainersNotReady: "containers with unready status: [svc-1]"
	ContainersNotReady: "containers with unready status: [svc-1]"
Waiting for pod gitlabcloud/runner-kgyedwqct-project-53701343-concurrent-0-wczv87xq to be running, status is Running
....
ERROR: Job failed (system failure): timed out waiting for pod to start

Environment description

Kubernetes Version: v1.30.3-eks-a18cd3a Gitlab-Runner Chart Version: 0.69.0

config.toml contents
[[runners]]
  environment = [
    "DOCKER_HOST=tcp://docker:2375",
    "DOCKER_TLS_CERTDIR=",
    "DOCKER_DRIVER=overlay2"
  ]
  [runners.kubernetes]
    privileged = true

    [[runners.kubernetes.services]]
      name = "public.ecr.aws/docker/library/docker:dind"
      alias = "docker"
      command = ["--tls=false", "--registry-mirror", "https://mirror.gcr.io"]

  [runners.machine]
    MachineOptions = [
      "engine-registry-mirror=https://mirror.gcr.io"
    ]