Investigate gitlab_connection_pool_size & gitlab_connection_pool_available_count metrics cardinality
One of our Customers reported a high cardinality problem with gitlab_connection_pool_size & gitlab_connection_pool_available_count metrics:
We found out that these two metrics have insane cardinality (>100k) for "gitlab-rails" (see screenshot):
- gitlab_connection_pool_size
- gitlab_connection_pool_available_count
Related MR: gitlab-org/gitlab!138340 (merged)
They later added:
These metrics have an high cardinality, meaning that they embed too many unique values of labels, you can read more about that over here.
After further investigation I've found out that for both metrics, the `pool_key` seems to be the faulty one for high cardinality: it has over 71k unique values.Does "pool_key="pool-15960", useful to investigate issues? With 100k+ metrics for the 2 labels, it requires a lot of resources to ingest all of that.