Instrument Job Inputs Interpolation

This issue used to be about measuring success of job inputs. That was mostly implemented by grouppipeline authoring in Track metrics when users manually provide job i... (#580221 - closed) and we'll use that to measure success. There is no way for a Runner to provide PID.

Expand to view original issue description for reference

Overview

This issue tracks adoption and usage metrics for the CI Job Inputs Interpolation feature to measure success post-release. These metrics will help evaluate feature adoption, validate business impact, and inform future iterations.

This is a prioritized list of usage data we must collect (originated from @dhershkovitch in &17833 (comment 2872038907)):

0. Percentage of pipelines that are using job inputs out of all running pipelines

Percentage of all executed CI pipelines that utilize the inputs keyword within the first 90 days after feature release.

1. Percentage of CI Jobs Using Inputs

Percentage of all executed CI jobs that utilize the inputs keyword within the first 90 days after feature release.

Measurement:

  • Track total CI jobs executed across GitLab (I expect this exists)
  • Track CI jobs with inputs defined in job payload
  • Calculate: (Jobs with inputs / Total jobs) × 100

2. Number of Projects Using CI Job Inputs

Unique count of projects that have executed at least one job with inputs.

Measurement:

  • Track unique project IDs where jobs with inputs have been executed

3. Percentage of Job Retries Using Modified Inputs

Of all job retries, what percentage are using modified input values versus retrying with the same inputs or no inputs.

Measurement:

  • Track total job retries
  • Track retries where input values differ from original job execution
  • Calculate: (Retries with modified inputs / Total retries) × 100

Apart from these 4 metrics it should be possible to group them (or even have individual metrics -> implementation detail) by namespace and user.

Instrumentation

In this iteration we want to provide the following prometheus metrics on the runner:

  • number of total interpolations
  • number of failed interpolations
  • number of how many inputs have been interpolated
Edited by Timo Furrer