Provides a method to take our rules and configure them inside of GKE
- Template script will convert the file into a PrometheusRules object
- Uses the filename to name the rule
- reads in the yaml file and templates it into the Kuberenetes Object
- Drops this into a new file which will only be loaded to validate it's proper yaml
- We perform two checks as part of CI to validate the file MAY be okay:
- validate yaml is legit yaml
- We ensure that the rules are created using strings
- We may still run into future situations where rules may contain invalid objects as we won't know failures until we watch the logs of the prometheus operator
- Applies a set of default labels for the rule selector to utilize: https://github.com/coreos/prometheus-operator/issues/2665#issuecomment-508941389
- To deploy, we'll rely on the Environments feature of GitLab and use a
CI/CD pipeline to apply to the correct environment
- Requires ops to be configured with the following ENV variables for
each Environment we would deploy too:
- PROJECT
- REGION
- CLUSTER
- SERVICE_KEY
- We use a simple for loop to iterate over the generated files to push them into kubernetes
- Requires ops to be configured with the following ENV variables for
each Environment we would deploy too:
- Modifies a few rules where validation indicated we need strings instead of floats or ints
- Since we need kubectl to apply these configurations, updates the Docker image to include the installation of said tool
Closes: https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7180
Edited by John Skarbek