Make labels order independent

This makes labels order independent in a way similar to what the official Prometheus::Client now does: https://github.com/prometheus/client_ruby/blob/7d9d45f547ce56ccf06344ab089e67228ddb7431/lib/prometheus/client/data_stores/direct_file_store.rb#L169

It shouldn't matter what order the labels were in for every change to the metric's value or how the metrics were initialized. But since the values were written out in 2 arrays before it did.

This works around that by sorting the label names and values alphabetically by label name before writing out the key of the metric into the file.

I discovered this while working on gitlab-org/gitlab!69154 (merged) for gitlab-com/gl-infra/scalability#1099

Questions remaining:

  1. Would this have been a problem for existing metrics that were initialized differently form the way they are counted?

    I don't think in our codebase it would make a big difference. We haven't done a lot of initialization in the past, other than the ones in the RequestsRackMiddleware. We should be running into the differently ordered labels there. But the source metric for these is hard to load in prometheus. These might indicate that some of the series remain empty. I don't think they should for that label combination, but there might be another reason those instances aren't receiving traffic. I'll look into that further.

  2. Do we want to fix this here? Or should I work around it in gitlab-org/gitlab!69154 (merged)? In my local environment, I was missing metrics, which is why I'd rather not just ignore the problem.

Edited by Bob Van Landuyt

Merge request reports

Loading