Track release-tools coordinated pipelines
What does this MR do and why?
This implements the first API scraper in delivery-metrics.
With this merge request we will start tracking the number, and status, of our coordinated pipelines.
# HELP delivery_deployment_pipelines_observer_errors Total number of errors from the pipelines observer
# TYPE delivery_deployment_pipelines_observer_errors counter
delivery_deployment_pipelines_observer_errors{reason="api_error"} 0
delivery_deployment_pipelines_observer_errors{reason="cache_failure"} 0
delivery_deployment_pipelines_observer_errors{reason="cache_full"} 0
# HELP delivery_deployment_pipelines_total Total number of coordinated pipelines
# TYPE delivery_deployment_pipelines_total gauge
delivery_deployment_pipelines_total{status="failed"} 0
delivery_deployment_pipelines_total{status="manual"} 2
delivery_deployment_pipelines_total{status="running"} 2
delivery_deployment_pipelines_total{status="scheduled"} 1
delivery_deployment_pipelines_total{status="success"} 1
delivery_deployment_pipelines_total{status="unknown"} 0
-
delivery_deployment_pipelines_observer_errors
tracks the observer failures -
delivery_deployment_pipelines_total
tracks the number of coordinated pipelines by the pipeline status.
Scraping happens in a dedicated goroutine and a read-only API token for OPS is needed as DELIVERY_METRICS_OPS_TOKEN
environment variable.
The scraping interval is 5 minutes and the observer is tolerant to 3 consecutive API failures, after that the cache is invalidated and we reset all the counters.
Author Check-list
-
Has documentation been updated?
Edited by Alessio Caiazza (OOO until 2024-08-26)