Attribute job analytics ClickHouse queries
What does this MR do and why?
Instruments the CI job analytics ClickHouse query so FinOps can attribute ClickHouse cost per root namespace. Attribution rides on the query log_comment, which carries root_namespace_id from the application context.
The wrinkle: the job analytics resolver does not execute the query. It returns a ClickHouseAggregatedRelation and defers execution to the GraphQL pagination connection. By execution time the resolver's local scope and any with_context block are gone, so the application context attributes have to be carried on the relation itself and applied at execution time.
Changes
Three edits:
lib/gitlab/graphql/pagination/click_house_aggregated_relation.rb: theSimpleDelegatorsubclass now takes a genericcontext_attributes:keyword in its constructor (default{}) and exposes it withattr_reader :context_attributes. The relation does not know about namespaces; it only carriesGitlab::ApplicationContextattributes for the deferred query. A real reader takes precedence over delegation, so pagination's delegated calls are unaffected. The default keeps the other caller,UserMetricsResolver(ClickHouseAggregatedRelation.new(result.payload)), unchanged.app/graphql/resolvers/ci/job_analytics_resolver.rb:resolvebuilds the relation withcontext_attributes: { namespace: project.project_namespace }.project_namespaceis a namespace record that is always present and hastraversal_ids, soroot_namespace_idresolves reliably.lib/gitlab/graphql/pagination/click_house_aggregated_connection.rb: inexecute_query(which this class already overrides), readitems.context_attributes; if present, wrap thesupercall inGitlab::ApplicationContext.with_context(context_attributes), otherwise callsuperunwrapped. Reading fromitems(the original relation) rather than the query argument is deliberate: pagination calls.having/.where/.limitwhich return a plainQueryBuilderwithout the reader, so onlyitemsstill carries it. Both the first and last pagination paths go throughexecute_query, so every pagination query is attributed.with_contextraisesArgumentErroron unknown attribute keys, so a typo fails loudly instead of silently dropping attribution.
Behavior and cost note: This change affects attribution only. Results are unchanged, and it adds no database queries. The resolver reads project.project_namespace, which Ci::JobAnalytics::QueryBuilder#execute already loads earlier in the same resolve call to scope the ClickHouse query by traversal_path. I measured this locally with ActiveRecord::QueryRecorder on the project(fullPath:) { jobAnalytics } request, using session and token auth with job_analytics_siphon on and off, and each request made exactly one project_namespace query, issued by existing code.
No feature flag (read-only, low risk). No Changelog trailer (internal observability).
References
- Related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/627521. This is part 3 of 3 stacked MRs (job analytics), the highest-scrutiny one since it touches shared pagination code.
- Follow-up issue for the same attribution on the AI metrics
user_metricsresolver (a sibling aggregated-relation resolver that stays unattributed, out of scope here): #630890. With this change the follow-up is a one-line change: passcontext_attributes:in that resolver too. - Attribution mechanism doc: https://docs.gitlab.com/development/database/clickhouse/clickhouse_within_gitlab/#query-attribution-with-log_comment
Screenshots or screen recordings
N/A - backend only, no UI changes.
How to set up and validate locally
-
Run the job analytics request spec (request specs need
VITE_ENABLED=true):VITE_ENABLED=true bundle exec rspec spec/requests/api/graphql/project/job_analytics_spec.rbExpected: 103 examples, 0 failures. Includes new examples asserting every executed ClickHouse query's
log_commentin the paginated request (both forward and backward pagination) carries the projectroot_namespace_id. -
Run both pagination connection specs:
bundle exec rspec spec/lib/gitlab/graphql/pagination/click_house_aggregated_connection_spec.rb spec/lib/gitlab/graphql/pagination/click_house_connection_spec.rbExpected: 51 examples, 0 failures. The aggregated connection spec has
#execute_queryunit tests: one asserts a subgroup attribution namespace resolves to the root group id, one asserts that with no attribution namespace the query runs without aroot_namespace_idkey. The plain connection spec proves the non-aggregated path is unaffected. -
Run the resolver spec:
bundle exec rspec spec/graphql/resolvers/ci/job_analytics_resolver_spec.rbExpected: 5 examples, 0 failures.
-
Run the AI user metrics request spec, which exercises the other
ClickHouseAggregatedRelationcaller with the defaultcontext_attributes(request specs needVITE_ENABLED=true):VITE_ENABLED=true bundle exec rspec ee/spec/requests/api/graphql/analytics/ai_analytics/ai_user_metrics_spec.rbExpected: 47 examples, 0 failures.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.