Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
GitLab
GitLab
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 36,990
    • Issues 36,990
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge requests 1,425
    • Merge requests 1,425
  • Requirements
    • Requirements
    • List
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Operations
    • Operations
    • Metrics
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI/CD
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.org
  • GitLabGitLab
  • Issues
  • #13203

Closed
Open
Created Aug 01, 2019 by Sarah Waldner@sarahwaldner✨Developer

Generic Alert Endpoint MVC

Problem to solve

Customers have many different tools they use to monitor their stack beyond Prometheus. We need to consume alerts from other sources. Since there are many monitoring tools that we could choose to integrate with, it would make more sense to create an endpoint/receiver/portal that customers can point their tools at a general portal that accepts alert payload.

Intended users

  • Delaney, Development Team Lead, https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead

  • Sasha, Software Developer, https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer

  • Devon, DevOps Engineer, https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer

  • Sidney, Systems Administrator, https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator

  • Sam, Security Analyst, https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst

Further Details

This supports the Incident Management vision.

Original Proposal

Create a REST endpoint that accepts payloads and creates an issue with the payload in the body of the issue. Designate fields that users can send in with the payload that map to the title of the issue and the summary. The summary is the section at the top of all issues that are created automatically from alerts. The summary includes:

  • title - This will be the title of the issue
  • starts_at - time the incident was triggered
  • description - high-level summary of the problem
  • service - affected service
  • runbook - associated run book

All of these fields will be optional from the user's perspective. If the user does not include the title field in the payload of the incoming alert, we need to figure out what the default title will be. If the user include any of the other fields (starts_at, description, service, runbook) in the payload of the incoming alert, we will exclude the field from the summary.

For the MVC, we will hard-code these fields and expect customers to configure alerts with values for these. Future iterations may include the ability for customers to choose what these fields are.

Questions:

  • What format do the payloads need to be?

Working MVC proposal

  • Receive alerts in the form of .json payloads to an alert endpoint (<project-path>/alerts/notify.json) as POST http method
  • GitLab will look for a field called title in the incoming .json. If it is not present, we will set the title of the issue to be New: Incident before creation and immediately following creation we will change it too New: Incident 123 - this will results in a system message posting to the incident timeline.
  • GitLab will also look for the following fields in the payload and paste them in the Summary section in the description of the issue in the order they are found in the payload. In any of the fields are not present, they will be excluded:
    • start_time
    • description
    • monitoring_tool
    • service
    • hosts
  • The entire payload will be posted in the issue discussion as a comment authored by the GitLab Alert Bot
  • Configuration for this endpoint will live under Settings > Integrations > Alert Endpoint (this will be a new option in the integrations list).

Designs

Designs showing how the payload would look if sent to the issue comments can be seen in Sketch Cloud.

Authentication will take place in Settings > Integrations, where it will live within the project services list. The copy for the 'Alert Endpoint' list item can be see here. Clicking on that list item, will take the user to an alert endpoint configuration page.

Permissions and Security

Documentation

Documentation required.

Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Links / references

  • Mural.ly storyboard
  • Mockups - Sketch Cloud

Current MRs

  1. Create a blank endpoint for generic alerts !15802 (merged)
  2. Add Settings page frontend !15733 (merged)
  3. Add generic alerts service & update project model - WIP !16117 (merged)
  4. Create issue from alert - !16021 (merged)
  5. Implement token-based authorization - #13203 (closed)
  6. Docs & Link to them from settings page - from !15733 (comment 213289744)
  7. Inline extra payload params !16441 (merged)
  8. Call NotifyService from NotificationsController !16508 (merged)
  9. Split alert parameters for the summary and details boxes #31883 (closed) !18261 (merged)
  10. Check Generic Alert payload size !16911 (closed)
  11. Add content-type to docs !17606 (merged)
  12. Fetch payload body from params[:notification] !17673 (merged)
  13. Skip CSRF protection in Generic Alerts Endpoint !17692 (merged)
  14. Add proper token check for Notify Service !17759 (merged)
  15. Remove feature flag !18570 (merged)
Edited Oct 16, 2019 by Vitali Tatarintev
Assignee
Assign to
12.4
Milestone
12.4 (Past due)
Assign milestone
Time tracking