Clickhouse GraphQL connection implementation for aggregations
What does this MR do and why?
Clickhouse GraphQL connection implementation for aggregations
References
Add jobAnalytics GraphQL API to fetch data on o... (#541703)
Parallel MR: Clickhouse model, finder support for job analytics (!204651 - merged)
Note:
Gem changes are added in the above MR, too. Whichever gets merged first will be rebased into the other. (So, please ignore this)
How to set up and validate locally
1. Setup Dependencies and Data
# Update gem dependencies
bundle install
# Verify the click_house-client gem is updated to 0.7.0
bundle list | grep click_house-client
2. Prepare test Data
Follow from MR !204651 (merged)
# Seed pipelines and jobs (triggers ClickHouse data insertion)
bundle exec rake "gitlab:seed:runner_fleet[root, rf-]"
3. Validation
# Rails console
# 1. Create test schema
test_schema = Class.new(GraphQL::Schema) do
use Gitlab::Graphql::Pagination::Connections
use BatchLoader::GraphQL
lazy_resolve ::Gitlab::Graphql::Lazy, :force
end
# 2. Create a simple query object for context
query_obj = OpenStruct.new(
schema: test_schema,
warden: GraphQL::Schema::Warden::PassThruWarden
)
# 3. Create context
context = GraphQL::Query::Context.new(
query: query_obj,
values: { connection: ClickHouse::Connection.new(:main) }
)
# 4. Build aggregated query - group by job name with mean duration
project_id = 1 # replace with your project ID
table = Arel::Table.new(:ci_finished_builds)
mean_duration = Arel::Nodes::Division.new(
Arel::Nodes::NamedFunction.new('avg', [table[:duration]]),
Arel::Nodes.build_quoted(1000.0)
).as('mean_duration_in_seconds')
aggregated_query = ClickHouse::Client::QueryBuilder
.new('ci_finished_builds')
.select(:name)
.select(mean_duration)
.where(project_id: project_id)
.group(:name)
.order(Arel.sql('mean_duration_in_seconds'), :desc)
# 5. Test the new aggregated relation wrapper
aggregated_relation = Gitlab::Graphql::Pagination::ClickHouseAggregatedRelation.new(aggregated_query)
# 6. Test GraphQL connection with pagination
connection = Gitlab::Graphql::Pagination::ClickHouseAggregatedConnection.new(
aggregated_relation,
context: context,
max_page_size: 5
)
# 7. Validate results
puts "Nodes: #{connection.nodes.inspect}"
puts "Has next page: #{connection.has_next_page}"
puts "Has previous page: #{connection.has_previous_page}"
# 8. Test cursor generation
if connection.nodes.any?
cursor = connection.cursor_for(connection.nodes.first)
puts "Sample cursor: #{cursor}"
# Test cursor decoding
decoded = JSON.parse(Base64.urlsafe_decode64(cursor))
puts "Decoded cursor: #{decoded.inspect}"
end
# 9. Test pagination with cursors
if connection.nodes.size > 1
# Test after cursor
after_cursor = connection.cursor_for(connection.nodes.first)
connection_after = Gitlab::Graphql::Pagination::ClickHouseAggregatedConnection.new(
aggregated_relation,
context: context,
after: after_cursor,
max_page_size: 3
)
puts "After cursor pagination - Nodes: #{connection_after.nodes.size}"
end
- Verify GraphQL Integration
# Rails console
test_relation = Gitlab::Graphql::Pagination::ClickHouseAggregatedRelation.new(
ClickHouse::Client::QueryBuilder.new('test')
)
connection_class = test_schema.connections.wrapper_for(test_relation)
puts "Connection class: #{connection_class}"
# Should output: Connection class: Gitlab::Graphql::Pagination::ClickHouseAggregatedConnection
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Narendran