Introduce gitlab_runner_process_state_info metric
What does this MR do?
Introduce gitlab_runner_process_state_info metric
This commit introduces a new Prometheus metric exported by the runner —
gitlab_runner_process_state_info one. It's always of value 1 and has
a state label defining one from four states the process can be at the
moment:
startingrunninggraceful-shutdownforceful-shutdown
It can be used to visualise not only what GitLab Processes are up (which
we can already do with many of our metrics, especially the
gitlab_runner_version_info one), but also in what state the processes
are in.
That's important in cases like the one described at gitlab-com/gl-infra/production-engineering#29100 (comment 3387635555)
Some of the metrics that GitLab Runner exposes are no more updated once
one of the shutdown modes is entered, while the last collected value is
still present and exported. The new gitlab_runner_process_state_info
metric allows to use the and feature of PromQL to selectively choose
which instances of GitLab Runner process should be use when handling
such metrics.