Support stage_name in CiJobAnalytics GraphQL API

What does this MR do and why?

  • Adds stage_name to group the CiJobAnalytics records
  • Removed existing stage object, which wouldn't return valid results on grouping (every pipeline creates a new stage)
  • No need to maintain backward compatibility as CiJobAnalytics API is experimental and also stage field is not used anywhere in the frontend code.

References

#580441 (comment 2959137261)

Screenshots or screen recordings

Screenshot_2025-12-19_at_10.33.12_PM

How to set up and validate locally

    1. Set up and Validate Test Data:
    require './spec/support/helpers/click_house_helpers.rb'
    
    include ClickHouseHelpers
    
    # 1. create test namespace and project
    user = User.first
    namespace = FactoryBot.create(:namespace, owner: user, path: "#{Time.now.to_i}-namespace-580441")
    project = FactoryBot.create(
      :project,
      name: "#{Time.now.to_i}-project-580441",
      path: "#{Time.now.to_i}-project-580441",
      namespace: namespace,
      owners: [user]
      )
    
    # 2. Create pipelines with stages and builds
    pipeline1 = FactoryBot.create(:ci_pipeline, project: project, started_at: Time.current, finished_at: Time.current + 10.seconds)
    pipeline2 = FactoryBot.create(:ci_pipeline, project: project, started_at: Time.current, finished_at: Time.current + 10.seconds)
    
    # 3. Create stages
    build_stage = FactoryBot.create(:ci_stage, pipeline: pipeline1, name: 'build')
    test_stage = FactoryBot.create(:ci_stage, pipeline: pipeline1, name: 'test')
    source_stage = FactoryBot.create(:ci_stage, pipeline: pipeline2, name: 'source-stage')
    ref_stage = FactoryBot.create(:ci_stage, pipeline: pipeline2, name: 'ref-stage')
    
    # 4. Create builds with stage_name
    build1 = FactoryBot.create(:ci_build, pipeline: pipeline1,  name: 'compile', ci_stage: build_stage, started_at: Time.current, finished_at: Time.current + 10.seconds)
    build2 = FactoryBot.create(:ci_build, pipeline: pipeline1,  name: 'unit-tests', ci_stage: test_stage, started_at: Time.current, finished_at: Time.current + 10.seconds)
    build3 = FactoryBot.create(:ci_build, pipeline: pipeline1,  name: 'integration-tests', ci_stage: test_stage, started_at: Time.current, finished_at: Time.current + 10.seconds)
    build4 = FactoryBot.create(:ci_build, pipeline: pipeline2,  name: 'source-job', ci_stage: source_stage, started_at: Time.current, finished_at: Time.current + 10.seconds)
    build5 = FactoryBot.create(:ci_build, pipeline: pipeline2,  name: 'ref-job', ci_stage: ref_stage, started_at: Time.current, finished_at: Time.current + 10.seconds)
    
    def expect(_anything) # mock method to ignore error in clickhouse_helpers
      true
    end
    def to(_any)
      true
    end
    def eq(_any)
      true
    end
    
    # 5. Insert builds to ClickHouse
    insert_ci_builds_to_click_house([build1, build2, build3, build4, build5])
    insert_ci_pipelines_to_click_house([pipeline1, pipeline2])
    
    # 6. Verify data was inserted
    puts "Builds inserted successfully!"
    puts "Project ID: #{project.id}"
    puts "Project PATH: #{project.full_path}"
    puts "Builds count: #{Ci::Build.where(project_id: project.id).count}"
    
    result = ClickHouse::Client.select(
      "SELECT DISTINCT stage_name FROM ci_finished_builds WHERE project_id = #{project.id}",
      :main
    )
    puts result.inspect
    
    # Should return: [{"stage_name"=>"build"}, {"stage_name"=>"test"}, {"stage_name"=>"source-stage"}, {"stage_name"=>"ref-stage"}]
    
  1. Validate the GraphQL results:
  • Open http://gdk.test:3000/-/graphql-explorer

  • Try executing using the below query and validate the results:

    {
      project(fullPath: "REPLACE_PROJECT.FULL_PATH") {
        id
        jobAnalytics(first: 20) {
          nodes {
            name
            stageName
            statistics {
              count
            }
          }
        }
      }
    }
  1. Cleanup the test data:

    ClickHouse::Client.execute(
      "ALTER TABLE ci_finished_builds DELETE WHERE project_id = #{project.id}",
      :main
    )
    
    # Verify deletion (rerun because if the return value is 5, because ClickHouse's `ALTER TABLE DELETE` is asynchronous and non-blocking.
    result = ClickHouse::Client.select(
      "SELECT COUNT(*) as count FROM ci_finished_builds WHERE project_id = #{project.id}",
      :main
    )
    puts "Remaining records: #{result.first['count']}"
    # Should return: 0
    
    project.destroy!
    namespace.destroy!
    

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Pedro Pombeiro

Merge request reports

Loading