Sync stage_name to ci_finished_builds clickhouse table
What does this MR do and why?
Sync stage_name to ci_finished_builds clickhouse table.
- Added
stage_namecolumn to clickhouse table in Add `stage_name` to `ci_finished_builds` ClickH... (!216825 - merged).
References
How to set up and validate locally
# 1. Create test data using factories
namespace = FactoryBot.create(:namespace, path: "#{Time.now.to_i}-namespace-580441")
project = FactoryBot.create(
:project,
name: "#{Time.now.to_i}-project-580441",
path: "#{Time.now.to_i}-project-580441",
namespace: namespace,
ci_delete_pipelines_in_seconds: 90.days.to_i
)
runner = FactoryBot.create(:ci_runner, :project, projects: [project])
runner_manager = FactoryBot.create(:ci_runner_machine, runner: runner, version: '16.4.0', revision: 'abc', platform: 'linux', architecture: 'amd64')
pipeline = FactoryBot.create(:ci_pipeline, project: project)
stage = FactoryBot.create(:ci_stage, pipeline: pipeline, name: 'deploy', position: 0)
build = FactoryBot.create(:ci_build, :success, pipeline: pipeline, runner_manager: runner_manager, ci_stage: stage)
# 2. Create a sync event for this build
event = Ci::FinishedBuildChSyncEvent.new(
build_id: build.id, project_id: build.project_id, build_finished_at: build.finished_at)
event.save!
# 3. Verify the service can load the build with stage_name
service = ClickHouse::DataIngestion::CiFinishedBuildsSyncService.new
scope = Ci::Build.id_in([build.id])
.left_outer_joins(:runner_manager, :ci_stage, runner: :owner_runner_namespace, project_mirror: :namespace_mirror)
.select(:finished_at, *service.send(:finished_build_projections))
loaded_build = scope.first
# 4. Verify stage_name is present
puts "Build ID: #{loaded_build.id}"
puts "Stage Name: #{loaded_build.ci_stage_name}"
puts "Expected: deploy"
puts "Match: #{loaded_build.ci_stage_name == 'deploy'}"
# 5. Verify CSV mapping includes stage_name
csv_mapping = ClickHouse::DataIngestion::CiFinishedBuildsSyncService::CSV_MAPPING
puts "\nCSV Mapping includes stage_name: #{csv_mapping.key?(:stage_name)}"
puts "stage_name maps to: #{csv_mapping[:stage_name]}"
# 6. Verify all expected fields are in the projection
projections = service.send(:finished_build_projections)
puts "\nProjections include ci_stage_name: #{projections.any? { |p| p.include?('ci_stage_name') }}"
# 7. Cleanup the data
runner.destroy!
runner_manager.destroy!
pipeline.destroy!
project.destroy!
namespace.destroy!
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Narendran