Add GraphQL lookahead optimization & percentile sort for job_analytics

What does this MR do and why?

This MR adds GraphQL lookahead optimization to the Job Analytics API and expands percentile sorting capabilities.

Key improvements:

  • Auto-detection of fields and aggregations: The resolver now uses GraphQL lookahead to automatically detect which fields and aggregations are requested in the query, eliminating the need for explicit selectFields and aggregations arguments (which will be removed in a follow-up MR).
  • Expanded percentile sorting: Added support for sorting by P50, P75, P90, and P99 percentiles in addition to the existing P95.
  • Simplified field naming: Renamed duration aggregation fields to shorter, more intuitive names (e.g., mean instead of mean_duration_in_seconds, p50 instead of p50_duration) to maintain consistent naming with PipelineAnalytics.

References

Related to #580442

Screenshots or screen recordings

Not applicable - backend API changes only

How to set up and validate locally

1. Database Preparation

Run the following rake task to seed pipelines and jobs in the monolith, which triggers a worker that inserts data into ClickHouse:

  bundle exec rake "gitlab:seed:runner_fleet[root, rf-]"

2. Validation

  1. Open GraphQL Explorer
  2. Use the following GraphQL query to validate the results:
{
  project(
    fullPath: "your-project-path"
  ) {
    jobAnalytics(
      nameSearch: "mock"
      sort: MEAN_ASC
      first: 20
      fromTime: "2025-01-01 00:00:00 UTC"
    ) {
      edges {
        node {
          name
          statistics {
            successRate: rate(status: SUCCESS)
            failureRate: rate(status: FAILED)
            success: count(status: SUCCESS)
            failure: count(status: FAILED)
            total: count
            durationStatistics {
              mean
              p50
              p95
            }
          }
        }
      }
    }
  }
}

3. Test different percentile sorts

Try sorting by different percentiles:

{
  project(fullPath: "your-project-path") {
    jobAnalytics(sort: P50_ASC) {
      nodes {
        name
        statistics {
          durationStatistics {
            p50
            p75
            p90
            p95
            p99
          }
        }
      }
    }
  }
}

Expected behavior:

  • The query automatically detects which fields and aggregations are needed based on the requested fields
  • No need to specify selectFields or aggregations arguments (will be removed in the next MR)
  • Results are sorted correctly by the specified percentile
  • All percentile values (P50, P75, P90, P95, P99) are available in the response (if selected)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Narendran

Merge request reports

Loading