Skip to content

Error uninstalling Prometheus: bad certificate

Summary

Cannot uninstall Prometheus 30 minutes after it is installed

Steps to reproduce

  1. Install Prometheus in a cluster
  2. Wait 30 minutes
  3. Uninstall

Example Project

https://staging.gitlab.com/hello-staging/minimal-ruby-app/clusters/1534

What is the current bug behavior?

While uninstalling, we cannot connect to Tiller. The issue is that the certificate in values-content-configuration-prometheus has expired

What is the expected correct behavior?

Prometheus uninstalls successfully (can connect to Tiller)

Relevant logs and/or screenshots

+ sleep 1s
Retrying (30)...
+ echo 'Retrying (30)...'
+ helm delete --purge prometheus --tls --tls-ca-cert /data/helm/prometheus/config/ca.pem --tls-cert /data/helm/prometheus/config/cert.pem --tls-key /data/helm/prometheus/config/key.pem
Error: remote error: tls: bad certificate
tkgl:gitlab tkuah$ kc get pods -n gitlab-managed-apps
NAME                                                     READY     STATUS    RESTARTS   AGE
uninstall-prometheus                                     0/1       Error     0          6m
$ kc get configmaps -n gitlab-managed-apps
NAME                                      DATA      AGE
values-content-configuration-prometheus   4         31d

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info

GitLab 12.0.pre

Results of GitLab application Check

Possible fixes

Update the config map in the uninstall method in https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/kubernetes/helm/api.rb#L25

Edited by Thong Kuah