Skip to content
GitLab Next
  • Menu
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • GitLab GitLab
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 43,831
    • Issues 43,831
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1,385
    • Merge requests 1,385
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.org
  • GitLabGitLab
  • Issues
  • #13401
Closed
Open
Created Aug 08, 2019 by Sarah Waldner@sarahwaldner🍉Developer

Automatically close GitLab issues with recovery alerts from Prometheus

Problem to solve

Incidents resolve for one of two reasons:

  1. Someone fixed the problem
  2. The problem fixed itself

In both scenarios, the monitoring tool often emits a recovery alert. In situations where a recovery alert is emitted, we want this to automatically close an associated incident in GitLab for the following reasons:

  1. If a system fixes itself, this may be unknown to the user, so we want the recovery alert to close the issue to indicate to responders that the incident is resolved.
  2. If someone fixes a problem and this is quickly recognized by the monitoring tool, closing an incident on a recovery alert saves the responder time.

Additionally, having this automation in place means that all users have the guarantee that open incidents are still active because once the problem has been solved they can rely on their tools to automatically close incident issues.

Intended users

Sasha the Software Developer
Devon the DevOps Engineer
Sidney the Systems Administrator

Further details

This work contributes to the Incident Management Vision

Proposal

We will need to store alert and incident data for this features. When the alert endpoint receives an alert of type=recovery, we will need to match that alert to an open issue on a unique identifier. Perhaps groupKey.

Permissions and Security

Documentation

Documentation required, most likely here.

Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Links / references

/label feature

Edited Oct 15, 2019 by Sarah Waldner
Assignee
Assign to
Time tracking