Add runner wait time stats by namespace
What does this MR do and why?
Adds runner wait time stats by namespace, it will be used in Visualize percentiles of estimated job wait tim... (#424790)
We already have very similar statistics available on the admin level. This MR adds:
- a
ci_finished_builds_aggregated_queueing_delay_percentiles_by_owner
table and materialized view to ClickHouse in order to store the aggregated percentiles for finished builds, by runner owner namespace. - a
GroupType.ciQueueingHistory
field that exposes the histograms from ClickHouse.
I tried to reuse what made sense.
Screenshots
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Follow https://docs.gitlab.com/ee/development/database/clickhouse/clickhouse_within_gitlab.html up until and including running clickhouse migrations.
- Create a group
- Create a group runner that belongs to that group
- Create a project within this group (use some template with the pipeline defined, e.g.
Pages - Plain HTML
- Make sure that the group runner is the only one available for this group
- Run the test pipeline in that project
- Wait until you have some builds finished (successfull, or failed)
- Wait another 2-5 minutes for builds to be synced to ClickHouse
- Go to
/-/graphql-explorer
in your GDK - Execute the following query (replace the group path):
{ group(fullPath: "my-group" ) { id ciQueueingHistory { timeSeries { p50 p75 p90 p95 p99 } } }}
- You should get a response similar to:
{ "data": { "group": { "id": "gid://gitlab/Group/93", "ciQueueingHistory": { "timeSeries": [ { "p50": 99.922, "p75": 99.922, "p90": 99.922, "p95": 99.922, "p99": 99.922 } ] } } } }
Edited by Pedro Pombeiro