Migrate `k8s-workloads/monitoring` to `gitlab-helmfiles` repo and convert to helmfile

Currently the repo at https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/monitoring/ manages the deployment and management of all our monitoring (mostly from the prometheus ecosystem) across all our GKE clusters

It manages the following helm releases and configurations/manifests

• alertmanager configuration from GKMS/GCS
• A helm release called gitlab-monitoring for the stable/prometheus-operator chart
• A raw deployment/service called thanos-query for some thanos-query extra components
• A raw deployment called cloudflare-exporter to deploy the cloudflare-exporter into gprd only

We want to migrate these manifests to be managed by helmfile in the gitlab-helmfiles repo.

To this end the work should encompass the following

• Migrate gitlab-monitoring

• Move the helm release gitlab-monitoring to helmfile control under gitlab-helmfiles

• Create a new helm release called gitlab-monitoring-secrets (using the incubator/raw chart) to take over management of the existing secrets needed for gitlab-monitoring

• Confirm and execute the steps needed to support the migration of the existing gitlab-secrets resources to helm control

• Confirm that, on a clean minikube install, in a clean checkout, helmfile -e pre apply correctly installs the gitlab-monitoring and gitlab-monitoring releases to the best we are currently able (GKE specific pieces will not work)

• Confirm that, a helmfile diff of gitlab-monitoring to the current installs in gstg, gprd, pre return no changes

• Confirm that once merged to master, a CI helmfile apply has run against all environments (not changing anything)

• Migrate thanos-query

• Create a new helm release called thanos-query (using the incubator/raw chart) to take over management of the existing thanos-query components

• Confirm that, on a clean minikube install, in a clean checkout, helmfile -e pre apply correctly installs the thanos-query release to the best we are currently able (GKE specific pieces will not work)

• Confirm and execute the steps needed to support the migration of the existing resources to helm control

• Confirm that, a helmfile diff of gitlab-monitoring to the current installs in gstg, gprd, pre return no changes

• Confirm that once merged to master, a CI helmfile apply has run against all environments (not changing anything)

• Migrate cloudflare-exporter

• Create a new helm release called cloudflare-exporter (using the incubator/raw chart) to take over management of the existing cloudflare-exporter deployment. This will include a secret called cloudflare-exporter which contains a username and API Key for accessing the cloudflare API`

• Confirm that, on a clean minikube install, in a clean checkout, helmfile -e pre apply correctly installs the cloudflare-exporter release to the best we are currently able (GKE specific pieces will not work)

• Confirm that, a helmfile diff of gitlab-monitoring to the current installs in gstg, gprd, pre return no changes

• Confirm that once merged to master, a CI helmfile apply has run against all environments (not changing anything)

We will also need to then do

• A commit to the k8s-workloads/monitoring repo announcing the decommission of the repo and where everything has moved