Skip to content

Regression: git TLS certificate verification fails with 10.1.0 or newer

Summary

When using a GitLab Runner version 10.1.0, cloning/fetching/etc. from our local GitLab instance fails with:

fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab.lan/infra/unifi.git/': SSL certificate problem: self signed certificate in certificate chain

The gitlab.lan host is using a signed certificate by a non-public CA, however the custom CA certificates are added to the Runner configuration.

Using any previous version of the Runner works with the exact same configuration (I'm actually using Docker, so it is just a matter of changing the runner tag version and recreateing the container to try it).

My Runner configuration is similar to:

concurrent = 1
check_interval = 0

[[runners]]
  name = "Docker Deploy Runner"
  url = "https://gitlab.lan/"
  token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
  tls-ca-file = "/etc/gitlab-runner/certs/ca.crt"
  executor = "docker"
  [runners.docker]
    image = "docker:latest"
    privileged = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
  [runners.cache]

The file ca.crt contains the custom CA chain.

Steps to reproduce

  1. Create a custom CA
  2. Create a certificate for your GitLab instance (example: gitlab.lan) and sign it with your custom CA
  3. Create a GitLab Runner using version <= 10.0.2
  4. Register it to your GitLab instance using the option tls-ca-file which points to the file with your custom CA chain
  5. Create a project which will use the runner and setup the CI
  6. Launch a CI pipeline: it works fine, the Runner picks the jobs, clone/fetch/etc. the repository, etc.
  7. Upgrade Runner to 10.1.0 and relaunch the pipeline --> it FAILS
  8. Downgrade to any previous versions --> it WORKS

Actual behavior

It is described within the steps. But to summarise:

  • Runner 10.1.0: fails with SSL certificate problem: self signed certificate in certificate chain
  • Runner <= 10.0.2: works fine

Expected behavior

It should be able to cclone/fetch, just like with previous release.

Relevant logs and/or screenshots

Runner's log:

Updating CA certificates...
Starting multi-runner from /etc/gitlab-runner/config.toml ...  builds=0
Running in system-mode.

Configuration loaded                                builds=0
Checking for jobs... received                       job=2013 repo_url=https://gitlab.lan/infra/unifi.git runner=8a9bb0f6
WARNING: Job failed: exit code 1                    job=2013 project=20 runner=8a9bb0f6

FAILED Job's log (release 10.1.0):

Running with gitlab-runner 10.1.0 (c1ecf97f)
  on Docker Deploy Runner (8a9bb0f6)
Using Docker executor with image registry.lan/infra/compose:1.16.1 ...
Using docker image sha256:c3fddb1b754804c51b659c8018d7ba6eb706f7580b2a49873777f53f4f87723c for predefined container...
Pulling docker image registry.lan/infra/compose:1.16.1 ...
Using docker image registry.lan/infra/compose:1.16.1 ID=sha256:f7b4c57f608e7e1f12703a79458212ce390d4609edc3b95490c00e5dff466122 for build container...
Running on runner-8a9bb0f6-project-20-concurrent-0 via runner.lan...
Cloning repository...
Cloning into '/builds/infra/unifi'...
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab.lan/infra/unifi.git/': SSL certificate problem: self signed certificate in certificate chain
ERROR: Job failed: exit code 1

SUCCESSFUL Job's log (previous releases):

Running with gitlab-runner 10.1.0 (c1ecf97f)
  on Docker Deploy Runner (8a9bb0f6)
Using Docker executor with image registry.lan/infra/compose:1.16.1 ...
Using docker image sha256:c3fddb1b754804c51b659c8018d7ba6eb706f7580b2a49873777f53f4f87723c for predefined container...
Pulling docker image registry.lan/infra/compose:1.16.1 ...
Using docker image registry.lan/infra/compose:1.16.1 ID=sha256:f7b4c57f608e7e1f12703a79458212ce390d4609edc3b95490c00e5dff466122 for build container...
Running on runner-8a9bb0f6-project-20-concurrent-0 via runner.lan...
Cloning repository...
Cloning into '/builds/infra/unifi'...
Checking out 5cde246a as master...
Skipping Git submodules setup
$ docker-compose up -d
Creating network "unifi_default" with the default driver
Creating unifi ... 
Creating unifi

Creating unifi ... done
Job succeeded

Environment description

This is on a custom installation using the Docker executor and Docker version 17.09.0-ce.

I create the gitlab-runner container using a Docker Compose file. If I use a version prior to 10.1.0 in the Compose file, everything is setup and works perfectly. If I just change the version in the compose to be 10.1.0, then it fails, no other changes are performed. This is easily reproducible by switching back and forth the version in the Compose file.

Used GitLab Runner version

Here is the version for the failing runners:

Version:      10.1.0
Git revision: c1ecf97f
Git branch:   10-1-stable
GO version:   go1.8.3
Built:        Sat, 21 Oct 2017 21:33:24 +0000
OS/Arch:      linux/amd64
Edited by J.-C. Berthon