x509: certificate is valid for ingress.local, not xxx.mydomain

Summary

Merging configuration from template file "/configmaps/config.template.toml" 
ERROR: Verifying runner... failed                   runner=3MEdYyzk5 status=couldn't execute POST against https://gitlab.mydomain.xxx/api/v4/runners/verify: Post "https://gitlab.mydomain.xxx/api/v4/runners/verify": tls: failed to verify certificate: x509: certificate is valid for ingress.local, not gitlab.mydomain.xxx
PANIC: Failed to verify the runner.                
Registration attempt 6 of 30
Runtime platform                                    arch=amd64 os=linux pid=149 revision=853330f9 version=16.5.0
WARNING: Running in user-mode.                     
WARNING: The user-mode requires you to manually start builds processing: 
WARNING: $ gitlab-runner run                       
WARNING: Use sudo for system-mode:                 
WARNING: $ sudo gitlab-runner...

Steps to reproduce

Install gitlab from helm, use custom domain. And create a new gitlab runner from GUI, get the token, fill it to the runner-token secret, and restart the gitlab-funner deployment.

Then there is the error.

Configuration used

Copy the secret gitlab-scm-wildcard-tls, to a new secret

kind: Secret
apiVersion: v1
metadata:
  name: gitlab-scm-runner-wildcard-tls
  namespace: git
  labels:
    app: gitlab
    chart: gitlab-7.6.0
    heritage: Helm
    release: gitlab-scm
data:
  gitlab.git.apricottavern.site.crt: >-
    LS0
  gitlab.git.apricottavern.site.key: >-
    LS0

Mount the secret

    spec:
      volumes:
        - name: runner-secrets
          emptyDir:
            medium: Memory
        - name: etc-gitlab-runner
          emptyDir:
            medium: Memory
        - name: projected-secrets
          projected:
            sources:
              - secret:
                  name: gitlab-scm-runner-wildcard-tls
              - secret:
                  name: gitlab-scm-minio-secret
              - secret:
                  name: gitlab-scm-gitlab-runner-secret
                  items:
                    - key: runner-registration-token
                      path: runner-registration-token
                    - key: runner-token
                      path: runner-token

(this is added)

              - secret:
                  name: gitlab-scm-runner-wildcard-tls

I already add

  config.template.toml: |
    [[runners]]
      tls-ca-file = "/secrets/gitlab.mydomain.xxx.crt"
      tls-cert-file = "/secrets/gitlab.mydomain.xxx.crt"
      tls-key-file = "/secrets/gitlab.mydomain.xxx.key"

to the k8s config map

I can confirm /secrets/gitlab.mydomain.xxx.crt exists by adding ls -al /secrets to config map

    echo "register crt from secrets"

    # file existes
    ls -al /secrets

    # this does not work, permission denied
    cp /secrets/xxx.crt
    /usr/local/share/ca-certificates/

    update-ca-certificates

  #  for i in $(seq 1 "${MAX_REGISTER_ATTEMPTS}"); do

Current behavior

runner can't register

Expected behavior

Use the crt specified in the tls-ca-file, and successfully register.

Versions

  • Chart: (tagged version | branch | hash git rev-parse HEAD) 0.59.1 16.6.0
  • Platform:
    • Cloud: (GKE | AKS | EKS | ?)
    • Self-hosted: (OpenShift | Minikube | Rancher RKE | ?) MicroK8S
  • Kubernetes: (kubectl version)
    • Client: v1.28.3
    • Server: v1.28.3
  • Helm: (helm version)
    • Client: v3.9.1
    • Server: v3.9.1

Relevant logs

(Please provide any relevant log snippets you have collected, using code blocks (```) to format)

this is now working

    echo "register crt from secrets"

    ls -al /secrets

    cp /secrets/gitlab.git.apricottavern.site.crt
    /usr/local/share/ca-certificates/

    update-ca-certificates

  #  for i in $(seq 1 "${MAX_REGISTER_ATTEMPTS}"); do

or

ls -al /etc/gitlab-runner/certs/

cp /secrets/gitlab.git.apricottavern.site.crt /etc/gitlab-runner/certs/

only getting errors like:

ls: /etc/gitlab-runner/certs/: Permission denied
cp: can't stat '/etc/gitlab-runner/certs/': Permission denied

this copy is succeed, but is not used too.

mkdir -p ~/.gitlab-runner/certs/

cp /secrets/gitlab.git.apricottavern.site.crt ~/.gitlab-runner/certs/
Edited by Ghost User