Setup dashboards endpoint and new common metrics yml schema
Problem to solve
Metrics Dashboard Framework - particularly https://gitlab.com/gitlab-org/gitlab-ce/issues/59974 and https://gitlab.com/gitlab-org/gitlab-ce/issues/58516, all rely on a few fundamental chunks of backend functionality.
We need:
- a new dashboards endpoint to expose the metrics dashboard layout
- a new schema for the default metrics that currently ship with gitlab
This issue represents the work to make the changes needed to distribute upcoming work on the metrics dashboard among multiple people. Any changes made as a part of this work should not impact product functionality.
Documentation
Active discussion on the technical considerations for the upcoming work can be found here: https://docs.google.com/document/d/1zQSUCO52V-2jaS-IoflHB_iTrJYWdzGD_ITn1mXMrWI/edit?usp=sharing
Testing
Break nothing.
What does success look like, and how can we measure that?
Once this item is complete, work should be able to continue on https://gitlab.com/gitlab-org/gitlab-ce/issues/59974 and https://gitlab.com/gitlab-org/gitlab-ce/issues/58516
Links / references
Defining Dashboards
Panel Properties:
- type* (Required)
- title (Required)
- y_label (Optional only for single-stat)
- presentation (Optional, used only in stacked-column-chart)
- metrics** (Required)
- ... (pass through all panel properties)
*Panel Types:
- area-chart
- single-stat
- line-chart
- column-chart
- stacked-column-chart
- heatmap
**Metric Properties:
- id (Optional, Required if alerting is required)
- query_range || query (Required)
- unit (Required)
- label (Optional)
- ... (pass through all metric properties)
**Optional Metric Properties:
- required_metrics ()
Sample Proposed YML Configuration
dashboard: 'Dashboard Title'
priority: 2 # order to appear in dashboard dropdown, higher priority should be higher in the dropdown
panel_groups:
- group: 'Group Title'
panels:
- type: area-chart # the type of chart to be used
title: "Chart Title"
y_label: "Y-Axis" # y-axis label
metrics:
- id: metric_of_ages # identifier used to join yml content to DB record
query_range: 'http_requests_total' # query or query_range, as should be passed to be prometheus endpoint
label: "Metric of Ages" # legend label for series
unit: "count" # unit for series data
What gets persisted?
Exactly the same as current metrics:
string title (dashboard.group)
string query (dashboard.group.panel.query or dashboard.group.panel.query_range)
string y_label (dashboard.group.panel.y_label)
string unit (dashboard.group.panel.metric.unit)
string legend (dashboard.group.panel.metric.label)
integer group (based on dashboard.group)
boolean common (true for GitLab-defined metrics across all projects)
string identifier (dashboard.group.panel.metric.id)
Refactor CommonMetricsImporter: Logic should remain mostly the same; changes are to keys or depth at which values are found. Metrics should only be persisted if an identifier is available; the content should simply be skipped if the field is not defined. ignore non-area charts unless environment_metrics_chart_type_options is enabled
Feature Flags
environment_metrics_use_prometheus_endpoint
-> Feature flag for enabling usage of the prometheus endpoint by the FE to dynamically fetch chart data. Determines which endpoint (new dashboards endpoint or additional_metrics) should be used for determining UI layout.
environment_metrics_show_multiple_dashboards
-> Feature flag for enabling multiple dashboards. Should control whether the metric dropdown is available. Requires environment_metrics_use_prometheus_endpoint
to be enabled.
environment_metrics_chart_type_options
-> Feature flag for utilizing non area-charts. Until on, only area charts are passed to the FE and FE assumes all panels are area-charts