Job execution hangs when latest docker:dind (version 19.03.6 and "floating" ones) is used

Summary

Gitlab pipeline timeout after given timeout seconds and saying ERROR: Job failed: execution took longer than 1h0m0s seconds because it is failing to pull image from gitlab/gitlab-runner-helper. Can you help as we are stuck here to deploy things live.

Workaround

Update your usage of docker:dind image to a specific version docker:19.03.5-dind for example:

image: docker:19.03.1
variables:
  # When using dind service, we need to instruct docker, to talk with
  # the daemon started inside of the service. The daemon is available
  # with a network connection instead of the default
  # /var/run/docker.sock socket. docker:19.03.1 does this automatically
  # by setting the DOCKER_HOST in
  # https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03.1/docker-entrypoint.sh#L23-L29
  #
  # The 'docker' hostname is the alias of the service container as described at
  # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services.
  #
  # Note that if you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier,
  # the variable must be set to tcp://localhost:2376 because of how the
  # Kubernetes executor connects services to the job container
  # DOCKER_HOST: tcp://localhost:2376
  #
  # Specify to Docker where to create the certificates, Docker will
  # create them automatically on boot, and will create
  # `/certs/client` that will be shared between the service and job
  # container, thanks to volume mount from config.toml
  DOCKER_TLS_CERTDIR: "/certs"
services:
  - docker:19.03.5-dind            # <------ We use a specific version of docker in docker
before_script:
  - docker info
build:
  stage: build
  script:
    - docker build -t my-docker-image .
    - docker run my-docker-image /script/to/run/tests

Root cause

• GitHub issues:
  • https://github.com/moby/moby/issues/40514
  • https://github.com/docker-library/docker/issues/216

As explained in https://github.com/moby/moby/issues/40514#issuecomment-586282433:

Our linux shared Runners use a single core machine to run the job, which explains why users on GitLab.com using our Runners expreienced this.

Preliminary testing looks to confirm that it's related to number of cores; single-core machines hit this issue, and 2 or more cores don't