ci_running_builds and ci_pending_builds metrics missing from gitlab-exporter in GitLab 13
Hi all,
We (being me and a colleague) found an issue in the metrics exporter of GitLab, after upgrading from GitLab CE 12.10 to GitLab CE 13.0. In Gitlab 12 this issue did not exist yet.
Issue
After upgrading GitLab to 13.0, the metrics ci_running_builds
and ci_pending_builds
are no longer available in the Prometheus scrape endpoint of the gitlab-exporter pod.
In the dashboard below we created it is clearly visible that after the update on 5/24 the metrics are no longer available:
What we checked
After querying the prometheus exporter scrape endpoint with Postman, we still see a ton of metrics in the response (around 630 or so including ci_stale_builds
and ci_unarchived_traces
), but the metrics ci_running_builds
and ci_pending_builds
are missing. We are aware that these are only ever available if they are not 0 so we kicked off a few pipelines in the background before querying to make sure they should be available.
We also checked if the query in
https://gitlab.com/gitlab-org/gitlab-exporter/-/blob/master/lib/gitlab_exporter/database/ci_builds.rb#L42-58 is still correct and when querying the database directly (with the placeholder %s
being replaced by running
) we indeed see a few running jobs for a few projects.
Expected behaviour
We would expect the values to still be available to us, even after upgrading to GitLab CE 13.0