Add a memory cache local to the thread to reduce Redis load
ApplicationSetting from Redis was responsible for at least 50% of the CPU load of the Redis cluster on GitLab.com. On GitLab.com, our serialized
ApplicationSetting is a whopping 54K due to the large number of
domain blacklists. Enabling compression for
Redis.cache would drop this down to 20K, but that is still large.
Since these settings generally don't change very much, we can load this from the database and
cache it in memory, skipping Redis altogether. We use
ActiveSupport::Cache::MemoryStore as a drop-in replacement for
RedisCacheStore even though we probably don't need synchronized access within
The main downside with this approach is cache invalidation: all Unicorn and Sidekiq processes cache their values for a minute before refreshing from the database. For routes under
/admin/application_settings, we adjust the controller to load the uncached values to prevent stale data from being inserted.
For the usage ping consent case, we also load an uncached value and tighten up the conditions when the query is run.
Closes #63977 (closed)