Upgrade monitoring helm charts
We want to upgrade the Prometheus helm chart so that we can leverage startupProbe
(corrective action for: gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!455 (comment 674101976) )
We should also upgrade any other charts that are out of date
-
Upgrade prometheus helm chart -
Upgrade prometheus-operator
topre
-
Update CRDs 👉 production#5699 (closed) -
Update helm chart 👉 gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!488 (merged)
-
-
Upgrade gstg
,ops
andorg-ci
-
Update CRDs 👉 production#5711 (closed) -
Update helm chart 👉 gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!493 (merged) -
Revert helm chart upgrade 👉 gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!494 (merged) -
Confirm that the fix in https://github.com/prometheus-operator/prometheus-operator/pull/4309 fixes the problem 👉 https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/13973#note_704600053 -
Wait for fixes upstream 👉 https://github.com/prometheus-operator/prometheus-operator/pull/4309 -
Wait for https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/13973#note_706320841 -
Upgrade upstream fixes in pre
👉 production#5740 (closed) -
Upgrade org-ci
andops
andgstg
👉 production#5731 (closed)
-
-
Upgrade gprd
-
First try (rolled back) 👉 production#5753 (closed) -
Fix kube-state-metrics
👉 production#5766 (closed) -
Second try 👉 production#5805 (closed)
-
-
-
Vendor kubernetes-incubator/raw similar to gitlab-com 👉 gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!491 (merged) -
Update bootstrap steps to use the current version that we are using 👉 gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!510 (merged) -
Stop using deprecated targetPort
👉 https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/13973#note_707205194. Moved to👉 https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/14458 -
Check if you can vendor the CRDs 👉 https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/14467 -
Fix breaking changes for kube-state-metrics
👉 https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/13973#note_710138098-
Enable boundaries for metrics -
Use kube_horizontalpodautoscaler_*
instead ofkube_hpa_*
👉 gitlab-com/runbooks!4017 (merged) -
Use kube_pod_container_[resource, allocatable, capacity]_[request,limits]{resource="X", unit="X"`` instead of
kube_pod_container_resource_requests_cpu_cores` and so on.👉 gitlab-com/runbooks!4022 (merged) -
Clean up metrics (remove metrics after upgrade) -
kube_hpa_*
👉 gitlab-com/runbooks!4027 (merged) -
Increase severity of kube_horizontalpodautoscaler_
alerts to match thekube_hpa
👉 gitlab-com/runbooks!4028 (merged)
-
-
Edited by Steve Xuereb