feat(pipelines): migrate analytics to the pipelines aggregation engine
What does this MR do and why?
Migrates pipeline analytics from the finishedPipelines aggregation engine to the new pipelines engine (gitlab!241321 (merged)), backed by siphon_p_ci_pipelines instead of the deprecated ci_finished_pipelines table. The new table holds all pipeline states, not just finished ones.
The GLQL query syntax is unchanged — existing queries keep compiling and running — but result semantics change: the engine now covers all pipeline states, so totalCount (and any query not filtered to terminal statuses) now includes running/pending pipelines. The rate metrics keep their old meaning: their denominator remains completed pipelines. Mechanically:
- The four rate metrics keep their names and compile to aliased selections of the engine's parameterised
outcomeRatemetric:successRate: outcomeRate(status: "success"). Response keys therefore still match the GLQL field names, so the transformer and thefieldsmetadata contract need no changes. - Sorting by a rate emits
orderBy: [{identifier: "outcomeRate", parameters: {status: "..."}}]via a newgraphql_sort_paramsanalyzer hook (default empty), consulted only when the sort field has no user-resolved parameters to inherit. Verified live that the backend disambiguates duplicateoutcomeRateorder identifiers by theirparameters. - The status filter widens from the four terminal statuses to the full CI status set (via the shared
ci_status_field_type()), reverting the !405 (merged) restriction —status = "running"analytics now work.
Docs update: gitlab!244089 (merged)
Agent update: gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!6116 (merged)
Example Usage
Build the Ruby extension once:
cd glql_rb
bundle install
bundle exec rake compileRate metrics with a rate sort (aliased outcomeRate selections)
mode: analytics
query: type = Pipeline and project = "gitlab-org/gitlab" and started > 2026-06-01
dimensions: ref
metrics: totalCount, successRate, failureRate, durationQuantile
sort: successRate descTest via ruby extension
GLQL_QUERY='type = Pipeline and project = "gitlab-org/gitlab" and started > 2026-06-01'
echo "q = Glql.compile('${GLQL_QUERY}', { mode: 'analytics', dimensions: 'ref', metrics: 'totalCount, successRate, failureRate, durationQuantile', sort: 'successRate desc' })[\"output\"]; puts q; File.write('/tmp/glql_query.json', { query: q, variables: { limit: 3 } }.to_json); exit" | bundle exec rake console && curl -s -X POST https://gitlab.com/api/graphql -H "Content-Type: application/json" -d @/tmp/glql_query.json | python3 -m json.toolGenerated GraphQL:
query GLQL($before: String, $after: String, $limit: Int) {
project(fullPath: "gitlab-org/gitlab") {
analytics {
pipelines(startedAtFrom: "2026-06-01 23:59") {
aggregated(before: $before, after: $after, first: $limit, orderBy: [{direction: DESC, identifier: "outcomeRate", parameters: {status: "success"}}]) {
count
nodes {
dimensions {
ref
}
totalCount
successRate: outcomeRate(status: "success")
failureRate: outcomeRate(status: "failed")
durationQuantile(quantile: 0.95)
}
}
}
}
}
}Response:
{
"data": {
"project": {
"analytics": {
"pipelines": {
"aggregated": {
"count": 26468,
"nodes": [
{
"dimensions": { "ref": "refs/workloads/affdbaf60ae" },
"totalCount": 1,
"successRate": 1.0,
"failureRate": 0.0,
"durationQuantile": 246.0
},
{
"dimensions": { "ref": "refs/workloads/429058c68c4" },
"totalCount": 1,
"successRate": 1.0,
"failureRate": 0.0,
"durationQuantile": 241.0
},
{
"dimensions": { "ref": "refs/workloads/34e106f4ac7" },
"totalCount": 1,
"successRate": 1.0,
"failureRate": 0.0,
"durationQuantile": 288.0
}
]
}
}
}
}
}
}Non-terminal status filter (new — impossible with finishedPipelines)
mode: analytics
query: type = Pipeline and project = "gitlab-org/gitlab" and status = "running"
dimensions: status, started(monthly)
metrics: totalCount
sort: started descTest via ruby extension
GLQL_QUERY='type = Pipeline and project = "gitlab-org/gitlab" and status = "running"'
echo "q = Glql.compile('${GLQL_QUERY}', { mode: 'analytics', dimensions: 'status, started(monthly)', metrics: 'totalCount', sort: 'started desc' })[\"output\"]; puts q; File.write('/tmp/glql_query.json', { query: q, variables: { limit: 3 } }.to_json); exit" | bundle exec rake console && curl -s -X POST https://gitlab.com/api/graphql -H "Content-Type: application/json" -d @/tmp/glql_query.json | python3 -m json.toolGenerated GraphQL:
query GLQL($before: String, $after: String, $limit: Int) {
project(fullPath: "gitlab-org/gitlab") {
analytics {
pipelines(status: ["running"]) {
aggregated(before: $before, after: $after, first: $limit, orderBy: [{direction: DESC, identifier: "startedAt", parameters: {granularity: "monthly"}}]) {
count
nodes {
dimensions {
status
startedAt(granularity: "monthly")
}
totalCount
}
}
}
}
}
}Response:
{
"data": {
"project": {
"analytics": {
"pipelines": {
"aggregated": {
"count": 44,
"nodes": [
{
"dimensions": { "status": "running", "startedAt": "2026-07-01" },
"totalCount": 53
},
{
"dimensions": { "status": "running", "startedAt": "2026-06-01" },
"totalCount": 6
},
{
"dimensions": { "status": "running", "startedAt": "2026-05-01" },
"totalCount": 9
}
]
}
}
}
}
}
}How to set up and validate locally
cargo test— full suite, including the new aliased-rendering, rate-sort, and non-terminal status filter cases intests/pipeline_analytics_tests.rs.DUMP_GRAPHQL=1 cargo test && npm run test:graphql— validates every generated query against the fetched GitLab schema (refresh it first with./scripts/fetch_gitlab_graphql_schema.sh).- Run either Example Usage snippet above against
gitlab.com/api/graphql.