Proposal: Add new metrics for pipelines and jobs and export metrics to OpenMetrics format (ideally) or Prometheus

Proposal

I am opening this feature proposal on behalf of a subscriber who would like to have a number of new metrics exported by GitLab. It would be ideal to export these metrics in OpenMetrics format but Prometheus would be acceptable.

Today, the requester is retrieving the requisite data via the GitLab API and converting it to Prometheus format and sending it to Prometheus. The goal of this feature proposal is to remove the requirement for an approach like this, which adds overhead.

The idea is that these metrics will be reported as time series data. These metrics fall into these two categories:

  • Pipelines
  • Jobs

Here's a list of the desired metrics, by category:

Pipelines

Metric 1: total number of pipelines per project

Total number of pipelines / project / timeseries: 1hr, 4hr, 8hr, 24hr -> specific interval is flexible.

  • Include project ID, project name, branch/tag/ref/kind, reference name, pipeline status (one of: created, waiting_for_resource, preparing, pending, running, success, failed, canceled, skipped, manual, scheduled)
  • Optionally: include all project info made available via the project API in timeseries format. This would be a collection of how that information looked at the point in time when the data was collected.

Metric 2: average number of pipelines per project per time series

Avg number of pipelines / project / timeseries: 1hr, 4hr, 8hr, 24hr -> specific interval is flexible.

  • Include project ID, project name, branch/tag/ref/kind, reference name, pipeline status (one of: created, waiting_for_resource, preparing, pending, running, success, failed, canceled, skipped, manual, scheduled)
  • Optionally: include all project info made available via the project API in timeseries format. This would be a collection of how that information looked at the point in time when the data was collected.

Metric 3: average pipeline duration per project per timeseries

Avg pipeline duration / project / timeseries: 1hr, 4hr, 8hr, 24hr -> specific interval is flexible.

  • Include project ID, project name, branch/tag/ref/kind, reference name, pipeline status (one of: created, waiting_for_resource, preparing, pending, running, success, failed, canceled, skipped, manual, scheduled)
  • Optionally: include all project info made available via the project API in timeseries format. This would be a collection of how that information looked at the point in time when the data was collected.

Jobs

Metric 4: Total number of jobs per project

Total number of jobs / project / timeseries: 1hr, 4hr, 8hr, 24hr -> specific interval is flexible.

  • Include job status
  • Include project ID, project name, branch/tag/ref/kind, reference name, pipeline status (one of: created, waiting_for_resource, preparing, pending, running, success, failed, canceled, skipped, manual, scheduled)

Metric 5: Average job duration per project

Avg job duration / project / timeseries: 1hr, 4hr, 8hr, 24hr -> specific interval is flexible.

  • Include job status
  • Include project ID, project name, branch/tag/ref/kind, reference name, status of pipeline that the job is running in (one of: created, waiting_for_resource, preparing, pending, running, success, failed, canceled, skipped, manual, scheduled)
Edited by Brie Carranza