feat(transform): accept the organization response root

Daniele Rossettirequested to merge
feat/151-organization-transform into main

Transform side of #151 (closed) (MR 2/2). MR 1, !497 (merged), is merged; this MR must ship in the same gem/npm release, since on MR 1 alone an organization-rooted response fails at transform.

What does this MR do and why?

MR 1 compiles group in (...) / project in (...) under the organization GraphQL root. The transformer unwraps the response by its root key and only knew namespace, project and group, so an organization-scoped response would have been reported as an invalid namespace. This MR:

  • recognises organization as a response root, keeping the existing precedence and the "empty root object is invalid" behaviour;
  • reports a null root as Organization does not exist or you do not have access to it, same wording as the other roots (a missing, empty or non-object root is attributed to the first root key present rather than always to namespace);
  • replaces the nested unwrap_or chain with a walk over ScopeKind::ALL (added here, with a completeness test in tests/derivable_apis_tests.rs), and drops InvalidScopeType, which had become a variant-for-variant copy of ScopeKind: InvalidScope now carries the ScopeKind and the message is derived from it.

Refused engine fields

When the backend refuses an analytics request it explains why in the GraphQL errors array (for example The following sources are not accessible: a, b) and nulls the engine field, or its aggregated connection when plan validation fails. Both consumers stop on errors (Apollo throws, /api/v4/glql returns a 400), so such a payload rarely reaches the transformer, but a caller that passed response["data"] on anyway used to get success: true with data: null. In analytics mode the transformer now:

  • reports a null engine field, or a null aggregated under it, by name: `duoCodeSuggestions` returned null, see `errors`.
  • never answers with a sibling engine's rows: a refused field wins over a present sibling whether the source is named or inferred, and a response without the named engine field is "no analytics data found" rather than probed for another engine.
  • leaves standard mode as it was: a null connection passes through, and an error there is a separate change.

Example Usage

cd glql_rb
bundle install
bundle exec rake compile

Same queries as MR 1's examples, now carried through the transform. Analytics data is private, so the snippets send GITLAB_TOKEN from the environment. Rows are from gitlab.com on 2026-09-07.

Aggregate across several groups

mode: analytics
query: type = CodeSuggestion and group in ("gitlab-org", "gitlab-com") and timestamp >= -30d
dimensions: language
metrics: totalCount, acceptanceRate
sort: totalCount desc
Test via ruby extension 
cd glql_rb # after the build prerequisite above

bundle exec ruby -Ilib - <<'RUBY'
require "gitlab_query_language"
require "json"
require "net/http"

query   = 'type = CodeSuggestion and group in ("gitlab-org", "gitlab-com") and timestamp >= -30d'
context = { mode: "analytics", dimensions: "language", metrics: "totalCount, acceptanceRate", sort: "totalCount desc" }

# 1. Compile GLQL -> GraphQL (also returns the typed `fields` the transform needs).
compiled = Glql.compile(query, context)
puts "== Generated GraphQL =="
puts compiled["output"]

# 2. Execute against GitLab. Analytics data is private, so send a token.
uri  = URI("https://gitlab.com/api/graphql")
http = Net::HTTP.new(uri.host, uri.port); http.use_ssl = true
req  = Net::HTTP::Post.new(uri, "Content-Type" => "application/json")
req["PRIVATE-TOKEN"] = ENV.fetch("GITLAB_TOKEN")
req.body = { query: compiled["output"], variables: { limit: 3 } }.to_json
response = JSON.parse(http.request(req).body)

# 3. Transform the response back into flat rows. Pass response["data"] (the
#    GraphQL `data` object, not the whole envelope) and reuse the compiled
#    `fields`, which carry the resolved parameters/aliases.
transformed = Glql.transform(response["data"], { mode: context[:mode], fields: compiled["fields"] })
puts "\n== Transformed rows =="
puts JSON.pretty_generate(transformed["data"])
RUBY
query GLQL($before: String, $after: String, $limit: Int) {
  organization {
    analytics {
      duoCodeSuggestions(descendantsScope: {groupFullPaths: ["gitlab-org", "gitlab-com"]}, timestampFrom: "2026-08-08 00:00") {
        aggregated(before: $before, after: $after, first: $limit, orderBy: [{direction: DESC, identifier: "totalCount"}]) {
          count
          nodes {
            dimensions {
              language
            }
            totalCount
            acceptanceRate
          }
        }
      }
    }
  }
}
{
  "count": 13,
  "nodes": [
    { "acceptanceRate": 0.04282744282744283, "totalCount": 2406, "language": "ruby" },
    { "acceptanceRate": 0.030821917808219176, "totalCount": 2339, "language": "" },
    { "acceptanceRate": 0.027901785714285716, "totalCount": 896, "language": "vue" }
  ]
}

Several groups, one row per project

mode: analytics
query: type = AgentPlatformSession and group in ("gitlab-org", "gitlab-com") and created >= -30d
dimensions: project
metrics: totalCount, usersCount
sort: totalCount desc
Test via ruby extension 
cd glql_rb # after the build prerequisite above

bundle exec ruby -Ilib - <<'RUBY'
require "gitlab_query_language"
require "json"
require "net/http"

query   = 'type = AgentPlatformSession and group in ("gitlab-org", "gitlab-com") and created >= -30d'
context = { mode: "analytics", dimensions: "project", metrics: "totalCount, usersCount", sort: "totalCount desc" }

# 1. Compile GLQL -> GraphQL (also returns the typed `fields` the transform needs).
compiled = Glql.compile(query, context)
puts "== Generated GraphQL =="
puts compiled["output"]

# 2. Execute against GitLab. Analytics data is private, so send a token.
uri  = URI("https://gitlab.com/api/graphql")
http = Net::HTTP.new(uri.host, uri.port); http.use_ssl = true
req  = Net::HTTP::Post.new(uri, "Content-Type" => "application/json")
req["PRIVATE-TOKEN"] = ENV.fetch("GITLAB_TOKEN")
req.body = { query: compiled["output"], variables: { limit: 3 } }.to_json
response = JSON.parse(http.request(req).body)

# 3. Transform the response back into flat rows. Pass response["data"] (the
#    GraphQL `data` object, not the whole envelope) and reuse the compiled
#    `fields`, which carry the resolved parameters/aliases.
transformed = Glql.transform(response["data"], { mode: context[:mode], fields: compiled["fields"] })
puts "\n== Transformed rows =="
puts JSON.pretty_generate(transformed["data"])
RUBY

Rows (project object truncated to fullPath):

{
  "count": 1199,
  "nodes": [
    { "usersCount": 52, "totalCount": 52453, "project": { "fullPath": "gitlab-org/editor-extensions/gitlab-lsp" } },
    { "usersCount": 45, "totalCount": 25392, "project": { "fullPath": "gitlab-org/ops/artifact-registry" } },
    { "usersCount": 799, "totalCount": 24684, "project": { "fullPath": "gitlab-org/gitlab" } }
  ]
}

A path the caller cannot read

mode: analytics
query: type = CodeSuggestion and group in ("gitlab-org", "no-such-group-glql-151") and timestamp >= -30d
dimensions: language
metrics: totalCount

The backend nulls the engine field and reports the reason in the GraphQL errors array. Both /api/glql and /api/v4/glql stop there; a caller that passes response["data"] to the transform anyway gets an error naming the field instead of success: true with data: null.

Test via ruby extension 
cd glql_rb # after the build prerequisite above

bundle exec ruby -Ilib - <<'RUBY'
require "gitlab_query_language"
require "json"
require "net/http"

query   = 'type = CodeSuggestion and group in ("gitlab-org", "no-such-group-glql-151") and timestamp >= -30d'
context = { mode: "analytics", dimensions: "language", metrics: "totalCount" }

# 1. Compile GLQL -> GraphQL.
compiled = Glql.compile(query, context)

# 2. Execute against GitLab. The unknown path makes the backend refuse the
#    engine field: it explains why in `errors` and nulls the field in `data`.
uri  = URI("https://gitlab.com/api/graphql")
http = Net::HTTP.new(uri.host, uri.port); http.use_ssl = true
req  = Net::HTTP::Post.new(uri, "Content-Type" => "application/json")
req["PRIVATE-TOKEN"] = ENV.fetch("GITLAB_TOKEN")
req.body = { query: compiled["output"], variables: { limit: 3 } }.to_json
response = JSON.parse(http.request(req).body)
puts "== errors =="
puts JSON.pretty_generate(response["errors"].map { |e| e["message"] })
puts "\n== data =="
puts JSON.pretty_generate(response["data"])

# 3. A consumer should stop at `errors`. One that transforms anyway now gets an
#    error naming the field instead of `success: true` with `data: null`.
transformed = Glql.transform(response["data"], { mode: context[:mode], fields: compiled["fields"] })
puts "\n== transform =="
puts JSON.pretty_generate(transformed.slice("success", "error", "data"))
RUBY
== errors ==
[
  "The following sources are not accessible: no-such-group-glql-151"
]

== data ==
{
  "organization": {
    "analytics": {
      "duoCodeSuggestions": null
    }
  }
}

== transform ==
{
  "success": false,
  "error": "Error: `duoCodeSuggestions` returned null, see `errors`.",
  "data": null
}

Checklist

  • cargo test (33 suites), cargo clippy --all-targets -- -D warnings, cargo fmt --check
  • Transform tests: organization-scoped response (inferred and named source, plus a compile-to-transform round trip for group in (...)), null organization root, null analytics wrapper, null engine field and null aggregated (by name), refused field next to a present sibling (named and inferred), absent named source
  • scope_kind_all_lists_every_variant guards ScopeKind::ALL against a future fifth root
  • Live run against gitlab.com through the Ruby extension (see Example Usage)
Edited by Daniele Rossetti

Merge request reports