praefect: Backport separate endpoint for datastore collector (v14.2)
1 unresolved thread
Compare changes
Files
4- John Cai authored
By default, when metrics are enabled, then each Praefect will expose information about how many read-only repositories there are, which requires Praefect to query the database. First, this will result in the same metrics being exposed by every Praefect given that the database is shared between all of them. And second, this will cause one query per Praefect per scraping run. This cost does add up and generate quite some load on the database, especially so if there is a lot of repositories in that database, up to a point where it may overload the database completely. Fix this issue by splitting metrics which hit the database into a separate endpoint "/db_metrics". This allows admins to set up a separate scraper with a different scraping interval for this metric, and furthermore it gives the ability to only scrape this metric for one of the Praefect instances so the work isn't unnecessarily duplicated. Given that this is a breaking change which will get backported, we must make this behaviour opt-in for now. We thus include a new configuration key "prometheus_use_database_endpoint" which enables the new behaviour such that existing installations' metrics won't break on a simple point release. The intent is to eventually remove this configuration though and enable it for all setups on a major release. Changelog: added (cherry picked from commit 7e74b733)
+ 31
− 5
@@ -63,11 +63,13 @@ import (
@@ -147,7 +149,9 @@ func main() {
@@ -190,7 +194,16 @@ func configure(conf config.Config) {
@@ -380,9 +393,18 @@ func run(cfgs []starter.Config, conf config.Config, b bootstrap.Listener, promre
@@ -403,9 +425,13 @@ func run(cfgs []starter.Config, conf config.Config, b bootstrap.Listener, promre