Skip to content

Instrument verification worker for observability

Sami Hiltunen requested to merge smh-verification-metrics into master

  • This commit adds metrics on the verification worker so the process becomes more observable. The metrics track the number of dequeued jobs per storage and then number of completed jobs per storage with their results. The number of stale leases releases is also tracked. This gives insight into how the verification work is progressing.

  • The verification worker is already collecting the metrics on how many jobs it's processing. Praefect is still lacking a metric for the overall verification queue depth which would be very helpful in determining how much work is there left for the verification to be completed. This commit adds a collector for the verification queue depth. The metric is collected for each storage separately. The metric differentiates between replicas that are unverified and replicas that have been verified but the verification has expired.

Closes #4097 (closed)

Merge request reports