Problem building with docker-in-docker using gitlab Registry Container with self-signed certificate

Hi, I have a problem with this configuration. It seems that from a dind container running from gitlab-runner I'm not able to run docker login against my gitlab Registry Container. It work if I run the docker login from the host (also from gitlab runner), from a docker with socket configuration or manually running the docker:dind container mounting local /etc/certs.d into the contained.

My configuration include: rhel7.2 gitlab 8.13.1 gitlab-ci-multi-runner-1.7.1-1.x86_64 docker-engine-1.12.3-1.el7.centos.x86_64

gitlab (and registry) are configured for https (with http redirection) with a self signed certificate on the same hostname (registry on port 5500)

On the gitlab-runner/docker server I've configured the certificate in the server trusted certificate and in the /etc/docker/certs.d/registryserverhost:5500 directory.

Running docker login work correctly from the host:

docker login registryserver:5500
Username (user):
Password: 
Login Succeeded

If i start a contained with docker:dind and dont' mount the /etc/docker/certs.d I get certificate error (same error I get from gitlab runner):

[root@server]# docker run --privileged -d docker:dind
c54ffb48a0e52c5180b436f001d060558cc3d567c123894bf8362ced22b99e1d
[root@server]# docker exec c54ffb48a0e52c5180b436f001d060558cc3d567c123894bf8362ced22b99e1d docker login  -u user -p password registryserver:5500
Error response from daemon: Get https://registryserver:5500/v1/users/: x509: certificate signed by unknown authority

If I mount the /etc/docker/certs.d into my docker:dind container it work correctly:

[root@server]# docker run -v /etc/docker/certs.d/:/etc/docker/certs.d --privileged -d docker:dind
7190ca51b735a302f86bf777761309206801d6398f1795474b1d038339bcfa3c
[root@server]# docker exec 7190ca51b735a302f86bf777761309206801d6398f1795474b1d038339bcfa3c docker login  -u user -p password registryserver:5500
Login Succeeded

I've configured the dind gitlab-runner as explained in this document: https://docs.gitlab.com/ce/ci/docker/using_docker_build.html I have this configuration in the runner (trying to mount the /etc/docker/certs.d directory as I'm doing manually but it don't work:

[[runners]]
  name = "dind"
  url = "https://server/ci"
  token = "token"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = true
    disable_cache = false
    volumes = ["/cache", "/etc/docker/certs.d:/etc/docker/certs.d"]
  [runners.cache]

My .gitlab-ci.yml script is something like this:

image: docker:latest
services: 
  - docker:dind
before_script:
  - docker info
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registryserver:5500
  
variables:
  CONTAINER_DIND_IMAGE: registryserver:5500/testgroup/testwebserver:dind

build_dind:
  stage: build
  script:
  - docker build  --pull -t $CONTAINER_DIND_IMAGE .
  - docker push $CONTAINER_DIND_IMAGE
  - echo "----------------------------------------"
  tags: 
  - dind

When it run I get the login error: $ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registryserver:5500 Error response from daemon: Get https://registryserver:5500/v1/users/: x509: certificate signed by unknown authority

Those are the information I got from the containers created by gitlab-runner (I've put a sleep command before the docker login so I can try to run commands manually). This is what I get:

List of the container running on the server (started from gitlab-runner)
[root@server]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
58c70f60ea41        1aed5271702d        "docker-entrypoint.sh"   8 seconds ago       Up 6 seconds                            runner-2c21ca3d-project-20-concurrent-0-build
3d8f8788e4f6        36ddb0269e3e        "dockerd-entrypoint.s"   12 seconds ago      Up 11 seconds       2375/tcp            runner-2c21ca3d-project-20-concurrent-0-docker

Running the docker login on the 2 container I get that docker daemon is running in one of them (the docker:dind ?) and not on the other (docker:latest ?) so on one I get the login error and on the other one i get the error that docker is not running on the host.

[root@server]# docker exec 58c70f60ea41 docker login  -u user -p password registryserver:5500
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[root@server]# docker exec 3d8f8788e4f6 docker login  -u user -p password registryserver:5500
Error response from daemon: Get https://registryserver:5500/v1/users/: x509: certificate signed by unknown authority

Checking if my directory is mounted I see that it is only on the machine where docker daemon is not running (probably it's correct as it's mounted on the docker:latest that is the image) and this create my problem.

[root@server]# docker exec 3d8f8788e4f6 ls -laR /etc/docker/
/etc/docker/:
total 8
drwx------    2 root     root            21 Oct 31 08:40 .
drwxr-xr-x   18 root     root          4096 Oct 31 08:40 ..
-rw-------    1 root     root           244 Oct 31 08:40 key.json
[root@server]# docker exec 58c70f60ea41 ls -laR /etc/docker/
/etc/docker/:
total 8
drwxr-xr-x    3 root     root            20 Oct 31 08:40 .
drwxr-xr-x   17 root     root          4096 Oct 31 08:40 ..
drwxr-xr-x    3 root     root          4096 Oct 30 17:15 certs.d

/etc/docker/certs.d:
total 8
drwxr-xr-x    3 root     root          4096 Oct 30 17:15 .
drwxr-xr-x    3 root     root            20 Oct 31 08:40 ..
drwxr-xr-x    2 root     root          4096 Oct 30 17:16 registryserver:5500

/etc/docker/certs.d/registryserver:5500:
total 12
drwxr-xr-x    2 root     root          4096 Oct 30 17:16 .
drwxr-xr-x    3 root     root          4096 Oct 30 17:15 ..
-rw-r--r--    1 root     root          1440 Oct 30 17:25 ca.crt

This is the docker info on the container where docker daemon is running:

[root@server]# docker exec 3d8f8788e4f6 docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.2
Storage Driver: vfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-327.18.2.el7.x86_64
Operating System: Alpine Linux v3.4 (containerized)
OSType: linux
Architecture: x86_64
CPUs: 2
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Total Memory: 15.67 GiB
Name: 3d8f8788e4f6
ID: SVA2:MG3R:74OZ:ZGBM:BMLU:ZY3U:EVZQ:4A5A:LJJL:5OK5:XEMF:DWMI
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8

I'm not sure I'm doing everything correctly but running the same integration on a runner defined as script or using docker-socket method described in the documentation works correctly. The only configuration not working is the one with docker-in-docker.

Can someone help me with this configuration?