Skip to content

Pipeline fails with "x509: certificate signed by unknown authority"

Summary

I have a self-hosted on-premise GitLab instance.

I successfully registered a runner on a new machine (setting Docker as the executor) by using sudo gitlab-runner register. Though, when I run the pipeline, it fails as soon as it tries to log in to the Docker registry, with the following error:

Error response from daemon: Get https://hostname.example:4994/v2/: x509: certificate signed by unknown authority

Certificate

I copied the certificate from my GitLab server (in /etc/gitlab/ssl/hostname.example.crt) to my new machine (in /etc/gitlab-runner/certs/hostname.example.crt).

Verifying the certificate

Running the following command on the new machine seems to work:

echo | sudo openssl s_client -CAfile /etc/gitlab-runner/certs/hostname.example.crt -connect hostname.example:443

The output (truncated for the sake of brevity) includes:

SSL handshake has read 1426 bytes and written 394 bytes
Verification: OK

This may indicate that the certificate is correctly installed on the machine, but the Docker executor is not using it.

Configuration

This is the configuration file that was generated in /etc/gitlab-runner/config.toml after registering the runner :

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "<name-example>"
  url = "<hostname.example>"
  token = "<token-example>"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0

Attempted solutions

I am aware that this error has been reported several times before (this may indicate that the docs need to be fixed/updated), but none of the proposed solutions seem to work. Here are some solutions that I tried:

  1. Setting volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] in config.toml.
  2. Setting volumes = ["/cache", "/etc/gitlab-runner/certs/hostname.example.crt:/etc/gitlab-runner/certs/hostname.example.crt:ro"] in config.toml.
  3. Merging the two settings of volumes in 1. and 2. to use both at the same time.
  4. Setting tls-ca-file = /etc/gitlab-runner/certs/hostname.example.crt in config.toml with 1.
  5. Setting GIT_SSL_NO_VERIFY: 'true' in the variables section of .gitlab-ci.yml.

Versions

Machine: Ubuntu 20.04.1 LTS

gitlab-runner: 13.8.0

docker: 19.03.8

Next?

What should I try next? I think it would be helpful if someone could provide a detailed list of all the steps that are necessary in order to set up a Docker executor that can correctly access a self-signed certificate.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information