Skip to content

Inject migration code path in HTTP metrics

João Pereira requested to merge 391-http into master

This MR adds the required changes to embed a new migration_path label in all registry_http_* Prometheus metrics so that we can partition them by code path (old, new, or none). The value for this label cannot be set at the handler level, it has to be set at the request level, because it varies per request.

So far we've been using gitlab.com/gitlab-org/labkit/metrics to automatically build/expose these HTTP metrics, which internally uses github.com/prometheus/client_golang/prometheus/promhttp(here). Unfortunatelly, promhttp only exposes code and method labels (plus any fixed labels, which we use for the route label here, defined once, when setting up each handler's instrumentation), it does not support dynamic/per-request labels. Because of this, it's not possible to achieve this without custom modifications.

promhttp is meant to be a generic/simple implementation for the most common scenario, so there is no desire to support dynamic/per-request labels like we're trying to achieve here (and the same was confirmed with the library authors in their mailing list here). Therefore, there is no point in changing Labkit to support something like this either, as there is no support for it in the underlying promhttp package.

Considering this, and because this is a specialized need, which we only need to keep around for the duration of the Phase 1 migration, the path of least resistance is to copy the source of the corresponding promhttp and labkit/metrics instrumentation methods and modify it so that it adds the label to each metric as we want to. This is far from ideal but it's the easiest way and the benefit from having this label during the migration is invaluable.

Related to #391 (closed)

Edited by João Pereira

Merge request reports