Skip to content

Backend: Add "POST #create" endpoint for webhooks to consume and create annotations

Summary

Users should be able to create annotations programmatically, to achieve that backend has to expose API endpoint which accepts token authorization. It should receive a POST request containing the annotations model as specified in https://gitlab.com/gitlab-org/gitlab/-/issues/211329#annotation-model and return JSON response with HTTP status indicating operation outcome.

How can webhooks use this API?

This API is similar to how chatops uses Grafana's API to create annotations.

Refer chatops code here

Click me view code/fold. 
# frozen_string_literal: true

module Chatops
  module Grafana
    class Annotate
      GRAFANA_URL = 'https://dashboards.gitlab.net'
      ANNOTATE_RETRIES = 3
      ANNOTATE_RETRY_INTERVAL = 3

      def initialize(token:)
        @token = token
      end

      def annotate!(text, tags: [])
        1.upto(ANNOTATE_RETRIES) do
          resp = HTTP
            .auth("Bearer #{@token}")
            .post(
              "#{GRAFANA_URL}/api/annotations",
              json: {
                text: text,
                tags: tags
              }
            )
          return if resp.status.ok?

          sleep ANNOTATE_RETRY_INTERVAL
        end
        raise "Failed to annotate after #{ANNOTATE_RETRIES} retries"
      end
    end
  end
end

Proposed Developer Workflow

  1. The user creates a token.
  2. Using the token a request is made to the annotations API endpoint to create new annotations.
  3. The request is made with the below-mentioned request params.
  4. The API result is returned as per the format specified below
  5. If an annotation is created, navigating to the monitoring dashboard should display annotation on all the charts (all charts for MVC).

Technical details

PoC available here: !27682 (diffs)

The params would include environment/cluster Id. (possibly supply these as env vars) Tags will be added in following interations

For the time being used in examples id is filename of a dashboard, but it may to be adjusted to be aligned with implementation #210287 (closed)

Requests

Environment metrics dashboard

POST /environments/:environment_id/metrics_dashboard/annotations/

Parameters:

Attribute Type Required Description
environment_id integer yes The ID of environment which dashboard needs to be annotated
dashboard_id dashboard_path string yes The ID of the dashboard which needs to be annotated
panel_id panel_xid string no The panle's ID to which annotation should be scoped
from starting_at string yes Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z. Timestamp marking start point of annotation
to ending_at string no Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z. Timestamp marking end point of annotation. When not supplied annotation will be displayed as single event at start point
description string yes Annotations description 
curl --header 'Private-Token: <your_access_token>' https://gitlab.example.com/api/v4/environments/1/metrics_dashboard/annotations \
 --data-urlencode "dashboard_path='.gitlab\dashboards\custom_metrics.yml'" \
 --data-urlencode "starting_at=2016-01-07T12:44:33.959Z" \
 --data-urlencode "description=annotation description"

Cluster metrics dashboard

For the time being custom dashboards are not available for clusters, because of that the only available id is id of a config/prometheus/cluster_metrics.yml

POST /clusters/:cluster_id/metrics_dashboard/annotations/

Parameters:

Attribute Type Required Description
cluster_id integer yes The ID of cluster which dashboard needs to be annotated
dashboard_id dashboard_path string yes The ID of the dashboard which needs to be annotated
panel_id panel_xid string no The panel's ID to which annotation should be scoped
from starting_at string yes Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z. Timestamp marking start point of annotation
to ending_at string no Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z. Timestamp marking end point of annotation. When not supplied annotation will be displayed as single event at start point
description string yes Annotations description 
curl --header 'Private-Token: <your_access_token>' https://gitlab.example.com/api/v4/clusters/1/metrics_dashboard/annotations \
 --data-urlencode "dashboard_path='.gitlab\dashboards\cluster_health.yml'" \
 --data-urlencode "starting_at=2016-01-07T12:44:33.959Z" \
 --data-urlencode "description=annotation description"

Response

On success:

HTTP status: 201

{
"message":"201 Created"
}

On error:

HTTP status: 400

{
  "message": { 
    "from": ["can't be blank"],
    "description": ["can't be blank"]
  }
}

@dbodicherla, thanks for the details summary. I believe that our focus for the MVC should be creating an automated annotation, so infra team can dogfood it, this is why

Frontend: Create a simple UI to create annotations. should probably be in the last iteration, this will also reduce some of the work as it would required design work

Edited by Mikołaj Wawrzyniak