migrate sd-exporter to kubernetes

It currently runs on: sd-exporter-01-inf-ops

It's cpu/memory saturated on occasions. There are issues with it: https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10936 . It's running on a legacy OS (Ubuntu 16.04).

Moving to k8s should help with updates of the exporter, updates of the underlying OS, observability of the exporter, ease of maintenance.

We will be migrating sd-exporter in the following environments

sd-exporter-01-inf-gstg.c.gitlab-staging-1.internal
sd-exporter-01-inf-ops.c.gitlab-ops.internal
sd-exporter-01-inf-gprd.c.gitlab-production.internal
sd-exporter-01-inf-db-benchmarking.c.gitlab-db-benchmarking.internal
sd-exporter-01-inf-pre.c.gitlab-pre.internal

Getting inspiration from previous similar deployment, I am going to evaluate if we can use the same plan:

gstg

  • Deploy Stackdriver GCP and K8s service accounts
  • Deploy Stackdriver exporter to K8s
  • Remove Stackdriver config and roles in chef-repo and gitlab-helmfiles repositories
  • Switch off Stackdriver VM
  • Delete Stackdriver VM

pre

  • Deploy Stackdriver GCP and K8s service accounts
  • Deploy Stackdriver exporter to K8s
  • Remove Stackdriver config and roles in chef-repo and gitlab-helmfiles repositories
  • Switch off Stackdriver VM
  • Delete Stackdriver VM

db-benchmarking

  • Deploy Stackdriver GCP and K8s service accounts
  • Deploy Stackdriver exporter to K8s
  • Remove Stackdriver config and roles in chef-repo and gitlab-helmfiles repositories
  • Switch off Stackdriver VM
  • Delete Stackdriver VM

ci

No migration needed as it is not being scraped at all.

  • Switch off Stackdriver VM
  • Delete Stackdriver VM

ops

  • Deploy Stackdriver GCP and K8s service accounts
  • Deploy Stackdriver exporter to K8s
  • Remove Stackdriver config and roles in chef-repo and gitlab-helmfiles repositories
  • Switch off Stackdriver VM
  • Delete Stackdriver VM

gprd

  • Deploy Stackdriver GCP and K8s service accounts
  • Deploy Stackdriver exporter to K8s
  • Remove Stackdriver config and roles in chef-repo and gitlab-helmfiles repositories
  • Switch off Stackdriver VM
  • Delete Stackdriver VM

other

  • Remove any remaining references to Stackdriver exporter and Prometheus in the Chef repo
Edited by Taliesin Millhouse