Skip to content

Implement service to generate CI runner usage report email

Pedro Pombeiro (OOO from July 16th till Aug 7th) requested to merge pedropombeiro/426560/runner-usage-analytics-export-service into master

What does this MR do and why?

This MR is the first MR of a break-up of the PoC MR. It adds the ability to generate a runner usage report that lists the top 500 projects that consumed runner minutes in the GitLab instance. The next MR will allow sending the report by email once ready. Each commit is reviewable individually.

Some context on the MR:

  • The CH table contains daily aggregations by project id, runner type, and job status.
  • We're currently reporting project aggregates, but we could easily filter/aggregate by runner type or job status in the future.
  • The report lists the top 500 projects with the most usage by descending order of total time - we can play with the 500 limit, for sure.
  • The time stamp we look at is the finished_at timestamp, since builds can have a started_at that is months behind the actual finish date (situation seen in production).

EE: true

Part of #425853 (closed)

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

These screenshots are not from this MR, but show the final state once the follow-up MR is in place:

Step Screenshot
GraphQL mutation image
E-mail image
Generated CSV image

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Go to the shell in your GDK gitlab directory and run bundle exec rake "gitlab:seed:runner_fleet". This will seed your GDK with some runners and jobs required for testing this MR.

  2. Run the following code in the GDK console:

    service = Ci::Runners::ExportUsageCsvService.new(current_user: User.find(1), runner_type: :instance_type, from_time: DateTime.new(2023)); service.execute

  3. Open http://gdk.test:3000/rails/letter_opener/ to see the generated email.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Pedro Pombeiro (OOO from July 16th till Aug 7th)

Merge request reports