User can define variables in the dashboard yml file using a PromQL query

Problem to solve

We need to be able to define variables in our dashboards. This feature is similar to variables in Grafana.

Intended users

Further details

In this issue, we will allow metric_label_values variables where the user defines a Prometheus series_selector whose result is used to populate the possible values for the variable.

Proposal

The following should be added to a dashboard in order for the dashboard to have template variables. The variables can be used in queries.

Note that the templating key will be a top-level dashboard property.

templating:
  variables:
    variable1:
      label: 'Variable 1'                                                      # (Optional) label that will appear in the UI for this dropdown.
      type: metric_label_values
      options:
        series_selector: 'backend:haproxy_backend_availability:ratio{env="{{env}}"}'
        label: 'backend'

• The values for the above variable1 will be all the different values for the backend label returned by the Prometheus series selector backend:haproxy_backend_availability:ratio{env="{{env}}"}.
• The series selector backend:haproxy_backend_availability:ratio{env="${env}"} requires the value of the env variable which means that the values for this variable need to be refreshed if the env variable value is changed. For the MVC, we plan to refresh the entire dashboard when a variable value is changed, so this should not be a problem.

Permissions and Security

Documentation

This needs to be documented at https://docs.gitlab.com/ee/user/project/integrations/prometheus.html#dashboard-yaml-properties with a warning that this is an alpha feature.

Something like this:

Availability & Testing

What does success look like, and how can we measure that?

Links / references

Spike to decide syntax: #201758 (closed)