Support alerting for dashboard yml-defined metrics (area-charts)

Problem to solve

Users should be able to set alerts on metrics defined in dashboard yml files in the project. The ability is available for gitlab-defined "common" metrics & custom metrics created in the UI, but not yet for yml-defined metrics. As a first step, we should look at bringing metrics defined for area-charts up to parity with existing metrics. Metrics for other panel types should come in another iteration.

• Open questions:
  • When do we want to persist metrics? Pipeline?
    • Should this step include validation & fail the build for invalid dashboards?
  • Do we want to clean-up project metrics which are no longer present in a dashboard? Or do they live forever?
  • Do we want to perform reconciliation between custom metrics created in the UI & dashboard yml-defined metrics?

Intended users

Further details

Proposal

Technical Implementation Proposal:

• Refactor CommonMetricsImporter to support project-defined metrics; Include a follow-up step that deletes removes project-defined metrics (probably distinguishing from custom metrics by the presence of the identifier from the yml)
• Call updated importer from pipeline? (This is the part I'm fuzzier on and has potential for scope creep)
• Update Metrics::Dashboard::Processor stages to account for project-defined dashboard metrics
• Test alerting (Should work out of the box, but we'll need to test)

First iteration:

• Should the persistence step include validation & fail the build for invalid dashboards? No.
• Do we want to clean-up project metrics which are no longer present in a dashboard? Or do they live forever? No, they are removed.
• Do we want to perform reconciliation between custom metrics created in the UI & dashboard yml-defined metrics? No. If a metrics has been defined in both spots, the user can do any cleanup they want to.

Permissions and Security

Documentation

Testing

What does success look like, and how can we measure that?

Links / references