This MR follows up on !165549 (merged) and enhances the Ci::CollectPipelineAnalyticsService
to allow calculating stats over a longer period of time, by selecting a different table for periods that exceed 1 week (aggregated daily instead of hourly in that case).
Closes #492116 (closed)
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
![]() |
![]() |
Numbered steps to set up and validate the change are strongly suggested.
Go to the shell in your GDK gitlab
directory and run bundle exec rake "gitlab:seed:runner_fleet"
. This will seed your GDK with some runners and jobs required for testing this MR.
In the GDK console, create ClickHouse sync events for the new pipelines:
Ci::FinishedPipelineChSyncEvent.insert_all(Ci::Pipeline.preload(:project).finished.where.not(finished_at: nil).order(finished_at: :asc).map { |pipeline| { project_namespace_id: pipeline.project.project_namespace_id, pipeline_id: pipeline.id, pipeline_finished_at: pipeline.finished_at, processed: false } }, unique_by: [:pipeline_id, :partition]);
In the GDK console, import the pipelines referred to by those sync events into ClickHouse:
Ci::ClickHouse::DataIngestion::FinishedPipelinesSyncService.new.execute
In http://gdk.test:3000/-/graphql-explorer, run the following script:
{
project(
fullPath: "rf-top-level-group-1/rf-group-1.1/rf-group-1.1.1/rf-project-1-1-1-1"
) {
id
pipelineAnalytics(fromTime: "2024-09-01", toTime: "2024-10-01") {
aggregate {
label
count
successCount: count(status: SUCCESS)
failedCount: count(status: FAILED)
otherCount: count(status: OTHER)
}
}
}
}
Adjust fromTime
and toTime
as needed. Before this MR, it wasn't possible to select a time window larger than 1 week, now you can go up to 1 year.