Embed logs in alerts from GitLab-managed Prometheus

Problem to solve

When triaging alerts, you need to be able to quickly investigate the issue, ideally without a lot of context switching, which costs you time that you can't afford to lose during an outage. By embedding the logs for a metric directly in an alert, GitLab saves you time by minimizing context switching. Logs can be accessed via logs tab in an alert.

Intended users

• Devon (DevOps Engineer)
• Allison (Application Ops)

Further details

This work supports the direction of the Alert Management product category.

Proposal

Embed the logs for the metric on which an alert was triggered in a new tab called Logs in Alert.

Design

Surface logs directly within a separate tab in the alerts themselves:

Second iteration - Embed Logs directly in Alert

• Note that this functionality will primarily benefit those people utilizing our Prometheus service. We can introduce the link to logs as long as the logs already exist within Gitlab.
• When we get to the point of adding a new tab to display logs: for alerts that originate outside of GitLab, we could add a field to the payload of the alert called log_url and a user could populate it if they want. We could then look for that link and surface it in the logs tab. Alternately, we could perhaps hide the link to the log tab for non-GitLab alerts.

Release Post Image

Permissions and Security

Documentation

Documentation required. Please add a new sub-section to this section.

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.