Add GraphQL lookahead optimization & percentile sort for job_analytics
What does this MR do and why?
This MR adds GraphQL lookahead optimization to the Job Analytics API and expands percentile sorting capabilities.
Key improvements:
- Auto-detection of fields and aggregations: The resolver now uses GraphQL lookahead to automatically detect which fields and aggregations are requested in the query, eliminating the need for explicit selectFields and aggregations arguments (which will be removed in a follow-up MR).
-
Expanded percentile sorting: Added support for sorting by
P50,P75,P90, andP99percentiles in addition to the existingP95. -
Simplified field naming: Renamed duration aggregation fields to shorter, more intuitive names (e.g.,
meaninstead ofmean_duration_in_seconds,p50instead ofp50_duration) to maintain consistent naming with PipelineAnalytics.
References
Related to #580442
Screenshots or screen recordings
Not applicable - backend API changes only
How to set up and validate locally
1. Database Preparation
Run the following rake task to seed pipelines and jobs in the monolith, which triggers a worker that inserts data into ClickHouse:
bundle exec rake "gitlab:seed:runner_fleet[root, rf-]"
2. Validation
- Open GraphQL Explorer
- Use the following GraphQL query to validate the results:
{
project(
fullPath: "your-project-path"
) {
jobAnalytics(
nameSearch: "mock"
sort: MEAN_ASC
first: 20
fromTime: "2025-01-01 00:00:00 UTC"
) {
edges {
node {
name
statistics {
successRate: rate(status: SUCCESS)
failureRate: rate(status: FAILED)
success: count(status: SUCCESS)
failure: count(status: FAILED)
total: count
durationStatistics {
mean
p50
p95
}
}
}
}
}
}
}
3. Test different percentile sorts
Try sorting by different percentiles:
{
project(fullPath: "your-project-path") {
jobAnalytics(sort: P50_ASC) {
nodes {
name
statistics {
durationStatistics {
p50
p75
p90
p95
p99
}
}
}
}
}
}
Expected behavior:
- The query automatically detects which fields and aggregations are needed based on the requested fields
- No need to specify selectFields or aggregations arguments (will be removed in the next MR)
- Results are sorted correctly by the specified percentile
- All percentile values (P50, P75, P90, P95, P99) are available in the response (if selected)
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Narendran