Skip to content

Provides a method to take our rules and configure them inside of GKE

  • Template script will convert the file into a PrometheusRules object
    • Uses the filename to name the rule
    • reads in the yaml file and templates it into the Kuberenetes Object
    • Drops this into a new file which will only be loaded to validate it's proper yaml
    • We perform two checks as part of CI to validate the file MAY be okay:
      • validate yaml is legit yaml
      • We ensure that the rules are created using strings
    • We may still run into future situations where rules may contain invalid objects as we won't know failures until we watch the logs of the prometheus operator
  • Applies a set of default labels for the rule selector to utilize: https://github.com/coreos/prometheus-operator/issues/2665#issuecomment-508941389
  • To deploy, we'll rely on the Environments feature of GitLab and use a CI/CD pipeline to apply to the correct environment
    • Requires ops to be configured with the following ENV variables for each Environment we would deploy too:
      • PROJECT
      • REGION
      • CLUSTER
      • SERVICE_KEY
    • We use a simple for loop to iterate over the generated files to push them into kubernetes
  • Modifies a few rules where validation indicated we need strings instead of floats or ints
  • Since we need kubectl to apply these configurations, updates the Docker image to include the installation of said tool

Closes: https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7180

Edited by John Skarbek

Merge request reports

Loading