Skip to content

Consider disabling prepared statements in Rails DB config

In an effort to see if I could figure out why Sidekiq was using a lot of RAM, I ran a ObjectSpace heap dump between 3 identical pushes in the Sidekiq task and did a diff analysis (see http://blog.skylight.io/hunting-for-leaks-in-ruby/). I noticed RAM usage of SIdekiq almost doubled after this even after running GC.start. The diff showed that a large amount of memory seemed to be in the statement cache:

Leaked 9 STRING objects of size 3264595/4922391 at: /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/arel-6.0.3/lib/arel/collectors/plain_string.rb:5
Leaked 9 STRING objects of size 3264730/4922691 at: /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/postgresql_adapter.rb:193

Sam Saffron also commented in https://github.com/rails/rails/issues/21992 here and disabled prepared statements in Discourse:

Keep in mind in many real world deployments there are 10 unicorns + 2 sidekiqs * 12 processes * 3-20 thread,, allowing 1000 statements adds up real fast to some very high numbers that live in database processes server side. As AR connection pools just keep connections open forever this quickly leads to memory running out. A 5 minute TTL cache would somewhat mitigate this issue. LRU redux has an efficient TTL cache that could be dropped in https://github.com/SamSaffron/lru_redux

See https://github.com/discourse/discourse/commit/dfe3ecb91412640ba94d0688d6fe9f5379b44610.

Should we consider having an option to disable this? Another thing to consider is that we can lower the default statement cache from 1000 to less.

/cc: @yorickpeterse, @pcarranza