docker login Error response from daemon: Client.Timeout exceeded while awaiting headers

Summary

I can be use docker login registry.example.com. But, it error in gitlab-ci.

Running with gitlab-runner 11.1.0 (081978aa)
  on docker-runner ee52c1d7
Using Docker executor with image docker:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:1e09f740386c351baa55960c86f34c7f64ca7ded041ea832eabe518d5c63c14b for docker:dind ...
Waiting for services to be up and running...
Pulling docker image docker:latest ...
Using docker image sha256:b22ef854da32fe8dafb2be7c6887b11d261f7b8567a6531f8d4a62502f998713 for docker:latest ...
Running on runner-ee52c1d7-project-20-concurrent-0 via 1ff8ad8ebe54...
Fetching changes...
HEAD is now at ef8cfd0 Update .gitlab-ci.yml
Checking out ef8cfd0d as v3...
Skipping Git submodules setup
$ apk update && apk add curl
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
v3.7.0-235-g12e79cd83d [http://dl-cdn.alpinelinux.org/alpine/v3.7/main]
v3.7.0-229-g087f28e29d [http://dl-cdn.alpinelinux.org/alpine/v3.7/community]
OK: 9048 distinct packages available
(1/3) Installing libssh2 (1.8.0-r2)
(2/3) Installing libcurl (7.61.0-r0)
(3/3) Installing curl (7.61.0-r0)
Executing busybox-1.27.2-r11.trigger
OK: 6 MiB in 17 packages
$ curl -Lv https://$CI_REGISTRY/v2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 172.17.0.2...
* TCP_NODELAY set
* Connected to registry.example.com (172.17.0.2) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [237 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2769 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=registry.example.com
*  start date: Jul 17 17:03:55 2018 GMT
*  expire date: Oct 15 17:03:55 2018 GMT
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET /v2 HTTP/1.1
> Host: registry.example.com
> User-Agent: curl/7.61.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Server: nginx
< Date: Sun, 05 Aug 2018 19:42:39 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 39
< Connection: keep-alive
< Docker-Distribution-Api-Version: registry/2.0
< Location: /v2/
< 
* Ignoring the response-body
{ [39 bytes data]

100    39  100    39    0     0    866      0 --:--:-- --:--:-- --:--:--   866
* Connection #0 to host registry.example.com left intact
* Issue another request to this URL: 'https://registry.example.com/v2/'
* Found bundle for host registry.example.com: 0x5595118dd500 [can pipeline]
* Re-using existing connection! (#0) with host registry.example.com
* Connected to registry.example.com (172.17.0.2) port 443 (#0)
> GET /v2/ HTTP/1.1
> Host: registry.example.com
> User-Agent: curl/7.61.0
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< Server: nginx
< Date: Sun, 05 Aug 2018 19:42:39 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 87
< Connection: keep-alive
< Docker-Distribution-Api-Version: registry/2.0
< Www-Authenticate: Bearer realm="https://gitlab.example.com/jwt/auth",service="container_registry"
< X-Content-Type-Options: nosniff
< 
{ [87 bytes data]

100    87  100    87    0  {"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}
   0   1705      0 --:--:-- --:--:-- --:--:--  1705
* Connection #0 to host registry.example.com left intact
$ docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://registry.example.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
ERROR: Job failed: exit code 1

Steps to reproduce

.gitlab-ci.yml

image: docker:latest

services:
  - docker:dind

variables:
  DOCKER_IMAGE: "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME"

before_script:
  - apk update && apk add curl
  - curl -Lv https://$CI_REGISTRY/v2
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY

build-latest:
  script:
    - docker pull $CI_REGISTRY_IMAGE || true
    - docker push $CI_REGISTRY_IMAGE
  only:
    - master
    
build:
  script:
    - docker pull $DOCKER_IMAGE || true
    - docker push $DOCKER_IMAGE
  only:
    - tags

config.toml

concurrent = 1
check_interval = 0

[[runners]]
  name = "docker-runner"
  url = "https://gitlab.example.com"
  token = "xxxxxxxxxxxxxxxxxxxxx"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:dind"
    privileged = true
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock:","/cache"]
    shm_size = 0
    links = ["gitlab:gitlab.example.com","gitlab:registry.example.com", "gitlab:page.example.com"]
  [runners.cache]
    Insecure = false

Actual behavior

$ docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://registry.example.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
ERROR: Job failed: exit code 1

Relevant logs and/or screenshots

image

Environment description

GitLab

GitLab CE Docker image: gitlab/gitlab-ce:11.1.4-ce.0

GITLAB_OMNIBUS_CONFIG: |
  external_url 'https://gitlab.example.com'
  registry_external_url 'https://registry.example.com'
  pages_external_url 'http://page.example.com'
  gitlab_pages['inplace_chroot'] = true
  letsencrypt['enable'] = true
  letsencrypt['contact_emails'] = ['xxx@example.com']
  gitlab_rails['gitlab_shell_ssh_port'] = 22
  gitlab_rails['omniauth_enabled'] = true

Docker

$ docker info
Containers: 4
 Running: 4
 Paused: 0
 Stopped: 0
Images: 17
Server Version: 17.12.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 89623f28b87a6004d4b785663257362d1658a729
runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-130-generic
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 2.938GiB
Name: gitlab
ID: FXNM:YD52:56SF:XZTN:4F2I:BNJX:3WKX:4VEB:WTAG:3FKT:LKGS:BCLW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

docker-compose.yml

version: '3.5'

services:
  gitlab:
    image: 'gitlab/gitlab-ce:11.1.4-ce.0'
    restart: always
    container_name: gitlab
    hostname: 'gitlab.example.com'
    extra_hosts:
      - "registry.example.com:127.0.1.1"
    network_mode: bridge
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.example.com'
        registry_external_url 'https://registry.example.com'
        pages_external_url 'http://page.example.com'
        gitlab_pages['inplace_chroot'] = true
        letsencrypt['enable'] = true
        letsencrypt['contact_emails'] = ['contact@example.com']
        gitlab_rails['gitlab_shell_ssh_port'] = 22
    ports:
      - '80:80'
      - '443:443'
      - '22:22'
    volumes:
      - '/srv/gitlab/config:/etc/gitlab'
      - '/srv/gitlab/logs:/var/log/gitlab'
      - '/srv/gitlab/data:/var/opt/gitlab'

  docker-runner:
    restart: always
    image: gitlab/gitlab-runner:latest
    network_mode: bridge
    links:
      - 'gitlab:registry.example.com'
      - 'gitlab:gitlab.example.com'
    volumes:
      - '/srv/runner/docker:/etc/gitlab-runner'
      - '/var/run/docker.sock:/var/run/docker.sock'

Used GitLab Runner version

Running with gitlab-runner
Version:      11.1.0
Git revision: 081978aa
Git branch:
GO version:   go1.8.7
Built:        2018-07-22T07:24:46+00:00
OS/Arch:      linux/amd64

Using Docker executor with image docker:latest
Edited by 🤖 GitLab Bot 🤖