Skip to content

Only create issues if supposed to for Prometheus alerts

Sarah Yasonik requested to merge sy-adhere-issue-creation-to-setting into master

What does this MR do?

When we receive an alert notification from Prometheus, we should only create an issue if the project is configured to do so. This MR brings the logic for Prometheus alert processing in line with the logic for generic alerts.

Related issue: #236180 (closed)

Screenshots

Screen_Shot_2020-09-08_at_6.39.03_PM

Testing

  1. Be a maintainer on a project

  2. Toggle the 'Create an issue' setting under Settings > Operations > Incidents to control whether an issue should be created Screen_Shot_2020-09-08_at_6.05.21_PM

  3. Enable alerting for 'External prometheus' under Settings > Operations > Alerts (and save changes)

  4. Trigger a test payload to prompt an alert to be created. Update the title & timestamp in this sample payload as-needed (alerts from prometheus are grouped by title & start time)

{
  "version" : "4",
  "groupKey": null,
  "status": "firing",
  "receiver": "",
  "groupLabels": {},
  "commonLabels": {},
  "commonAnnotations": {},
  "externalURL": "", 
  "alerts": [{
    "startsAt": "2020-09-08T11:22:40Z", 
    "generatorURL": "http://host?g0.expr=up", 
    "endsAt": null,
    "status": "firing",
    "labels": {
      "gitlab_environment_name": "production"
    }, 
    "annotations": {
      "title": "Here is an alert", 
      "gitlab_y_label": "status"
    }
  }]
}
  1. See the alerts under Operations > Alerts to see the automatically created (or not) issues

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Sarah Yasonik

Merge request reports