Commit 1f13ab57 authored by John Skarbek's avatar John Skarbek

Merge branch 'jarv/add-omnibus-alerts' into 'master'

Add alert for omnibus package

See merge request !1080
parents 1c3505cb 03a2d40b
groups:
- name: omnibus.rules
rules:
- alert: OmnibusPackageUpdatesDisabled
expr: count(omnibus_package_install{enable="true"}) / count(omnibus_package_install) < .25
for: 6h
labels:
severity: warn
annotations:
description: |
At least 75% of the fleet have omnibus package updates disabled.
This prevents automatic package installations for newly provisioned
VMs
runbook: troubleshooting/omnibus-package-updates.md
title: Omnibus package updates have been disabled for over 6 hours
# Omnibus package troubleshooting
## First and foremost
*Don't Panic*
## Symptoms
* Omnibus packages are set to disabled preventing automatic updates
## Troubleshooting
Check the omnibus role for the environment
```
# Example for the gprd environment
knife role show gprd-omnibus-version
```
See whether the `enable` attribute is set to `true`.
If it is not, ensure there are not any recent failed deployments by checking the
`#announcements` slack channel or recent [deployer pipelines](https://ops.gitlab.net/gitlab-com/gl-infra/deployer/pipelines)
If unsure, contact someone in the delivery team in `#g_delivery` about the error so they can
investigate.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment