Skip to content

GraphQL: Allow periods longer than one week for PipelineAnalytics

Pedro Pombeiro (OOO on the week of Oct 14th-18th) requested to merge pedropombeiro/492116/allow-periods-longer-than-1-week into master

What does this MR do and why?

This MR follows up on !165549 (merged) and enhances the Ci::CollectPipelineAnalyticsService to allow calculating stats over a longer period of time, by selecting a different table for periods that exceed 1 week (aggregated daily instead of hourly in that case).

Closes #492116 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After
image image

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Go to the shell in your GDK gitlab directory and run bundle exec rake "gitlab:seed:runner_fleet". This will seed your GDK with some runners and jobs required for testing this MR.

  2. In the GDK console, create ClickHouse sync events for the new pipelines:

    Ci::FinishedPipelineChSyncEvent.insert_all(Ci::Pipeline.preload(:project).finished.where.not(finished_at: nil).order(finished_at: :asc).map { |pipeline| { project_namespace_id: pipeline.project.project_namespace_id, pipeline_id: pipeline.id, pipeline_finished_at: pipeline.finished_at, processed: false } }, unique_by: [:pipeline_id, :partition]);

  3. In the GDK console, import the pipelines referred to by those sync events into ClickHouse:

    Ci::ClickHouse::DataIngestion::FinishedPipelinesSyncService.new.execute

  4. In http://gdk.test:3000/-/graphql-explorer, run the following script:

    {
  project(
    fullPath: "rf-top-level-group-1/rf-group-1.1/rf-group-1.1.1/rf-project-1-1-1-1"
  ) {
    id
    pipelineAnalytics(fromTime: "2024-09-01", toTime: "2024-10-01") {
      aggregate {
        label
        count
        successCount: count(status: SUCCESS)
        failedCount: count(status: FAILED)
        otherCount: count(status: OTHER)
      }
    }
  }
}

Adjust fromTime and toTime as needed. Before this MR, it wasn't possible to select a time window larger than 1 week, now you can go up to 1 year.

Edited by Pedro Pombeiro (OOO on the week of Oct 14th-18th)

Merge request reports