chore: upgrade prom chart 19.1.0 pre
Background
This is the second attempt to upgrade the Prometheus chart and Operator. The first attempt had to be reverted.
We need to upgrade to newer versions of the CRDs and helm chart so that
we can specify startupProbe
for a corrective
action.
Solution
Upgrade the operator and chart version to all environments except
gprd
.
Specify a version of the Prometheus operator that supports our alertmanager configuration, specifically this commit we can that that that image is using the correct version by running the commands below.
$ docker run --rm -it ghcr.io/prometheus-operator/prometheus-operator:master@sha256:bb79240165868c7d73d3db2b45bd065bf2b3050729aa4809f6de79cace232feb --version
prometheus-operator, version 0.51.2 (branch: master, revision: 0a38647379a5e93f639bf8e634deabcc32e01fb6)
build user: Action-Run-ID-1345540349
build date: 20211015-10:23:54
go version: go1.17.2
platform: linux/amd64⏎
Use the image from GitHub Container registry since quay.io has a small retention policy. We can this specific digest was pushed looking at GitHub Actions for that commit.
Upgrade
In gitlab-com/gl-infra/production#5711 (closed) we
updated the CRDs for v0.50.0
however, now we are using a newer version
of the operator so we need to use the CRDs available in
0a38647379a5e93f639bf8e634deabcc32e01fb6
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagerconfigs.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_probes.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_prometheusrules.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/0a38647379a5e93f639bf8e634deabcc32e01fb6/example/prometheus-operator-crd/monitoring.coreos.com_thanosrulers.yaml
reference https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/13973