gitlab-runner-helper: entrypoint not executed

Summary

For some reason, the gitlab-runner-helper image does not start its /entrypoint, but the normal gitlab-runner image does. According to https://docs.gitlab.com/runner/configuration/tls-self-signed.html#trusting-the-certificate-for-the-other-cicd-stages it should "just work".

Related issues

Possibly lots more with similar symptoms.

Steps to reproduce

I'm using my own CA and my own GitLab instance. My own runner setup is as follows:

  • gitlab-runner is as container with the Docker socket passed as volume
  • I've put my gitlab-runner config in /srv/gitlab-runner/config and bind mount that path into the gitlab-runner container via /srv/gitlab-runner/config:/etc/gitlab-runner/certs/ca.crt
  • config.toml is described below
.gitlab-ci.yml The CI YAML is not the problem. I'm using an S3 cache which works when I bake my CA file into a custom helper image.

Actual behavior

Using my S3 cache with the official and unmodified gitlab-runner-helper image:

WARNING: Retrying...                                error=Get "https://<my-s3-url-with-certificate-from-custom-CA>/runner/project/663/master-protected?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=.....-Amz-Date=20230829T085605Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=...X-Amz-Signature=...": x509: certificate signed by unknown authority

When I run the gitlab-runner-helper image manually (both version 14.9.2 and 16.3.0), nothing happens:

# v14.9.2
$ docker run --rm -v /srv/gitlab-runner/config/certs/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro -it registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:alpine3.15-x86_64-v14.9.2 sh
/ #

# v16.3.0
$ docker run --rm -v /srv/gitlab-runner/config/certs/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro -it registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:alpine3.18-x86_64-v16.3.0 sh
/ #

Expected behavior

The docker run above should run the /entrypoint script and update the CA certificates automatically, as far as I can tell from the source code for v14.9.2 and v16.3.0. But it doesn't.

Manually executing it does the trick:

$ docker run --rm -v /srv/gitlab-runner/config/certs/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro -it registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:alpine3.18-x86_64-v16.3.0 sh
/ # /entrypoint
Updating CA certificates...
WARNING: ca-cert-ca.pem does not contain exactly one certificate or CRL: skipping

From the docker run behavior, I suspect that the same thing happens during CI pipelines: /entrypoint does not execute for the helper image, but for the normal image it does. I can openssl s_client -connect ${CI_SERVER_HOST}:443 just fine.

Relevant logs and/or screenshots

None.

Environment description

Using my own CA, my own GitLab instance, my own shared runners.

config.toml contents 
...

[[runner]]
  ...
  tls-ca-file = "/etc/gitlab-runner/certs/ca.crt"
  ...
  [[runners.docker]]
    ...
    volumes = ["/srv/gitlab-runner/config/certs/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro"]
    ...

Used GitLab Runner version

Both 14.9.2 and 16.3.0.

Edited by thetredev