Skip to content

Add caching minimum/maximum ids in new instrumentation classes for batch queries

What does this MR do?

Related to #326338 (closed)

This adds caching minimum/maximum ids for batch queries in new instrumentation database metrics, in order to memoize minimum and maximum ids for big tables and calculate them only once. A similar approach was used in Gitlab::UsageData.

Caching is done only by instrumentation classes that explicitly pass a cache key using cache_start_and_finish_as, to prevent unintended pollution.

Redis caching through Rails.cache was used for simplicity:

  • The ids would have to be shared between different subclasses of DatabaseMetric. Class variables are not a viable option, as they might have some unexpected side effects on production, where classes are not reloaded and this data might persist between Usage Ping runs.
  • We'll eventually want to run new instrumentation Usage Ping metrics in parallel (e.g. using Sidekiq). In that case, simple memoization won't be useful, as Sidekiq jobs don't share memory and the only option will be to use an external database anyway.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Piotr Skorupa

Merge request reports