Skip to content
  • Julian Maurice's avatar
    Bug 28306: Allow to query database with minimal memory footprint · 4605304f
    Julian Maurice authored and joubu's avatar joubu committed
    
    
    The goal is to be able to build a database handler (dbh) and to execute
    queries without loading unnecessary stuff. This will be useful to reduce
    memory usage of daemons that need to check the database
    periodically
    
    The patch provides a new method Koha::Database::dbh which returns a
    database handler without loading the DBIx::Class schema. This method is
    also used by DBIx::Class, so whether you use DBI or DBIx::Class, the
    same method is used to initialize the connection.
    
    The patch also moves some code in order to avoid loading C4::Context:
    - C4::Context::timezone moves to Koha::Config
    - C4::Context::db_scheme2dbi moves to Koha::Database
    
    To measure memory usage I used the following commands:
    
    * before the patch:
    perl -MKoha::Database \
        -E 'Koha::Database->schema->storage->dbh->do("select 1");' \
        -E '$|=1; say $$; sleep 2' \
        | while read pid; do ps -p $pid -o rss=; done
    
    * after the patch:
    perl -MKoha::Database \
        -E 'Koha::Database->dbh->do("select 1");' \
        -E '$|=1; say $$; sleep 2' \
        | while read pid; do ps -p $pid -o rss=; done
    
    It will give you the RSS (Resident Set Size) of the perl process in kB
    
    What I get:
    * before the patch: between 96.9MB and 97.2MB
    * after the patch: between 17.8MB and 18.2MB
    
    Note that if a timezone is configured (either from $KOHA_CONF or
    TZ environment variable), Koha will load DateTime::Timezone to check if
    it's valid, and it increases RSS to 36MB
    
    Another interesting metric is the number of modules loaded:
    * before the patch:
    perl -MKoha::Database \
        -E 'Koha::Database->schema->storage->dbh;' \
        -E 'say scalar keys %INC'
    
    Result: 567
    
    * after the patch:
    perl -MKoha::Database \
        -E 'Koha::Database->dbh;' \
        -E 'say scalar keys %INC'
    
    Result: 51
    
    Test plan:
    1. Apply the patch & restart starman
    2. Make sure Koha is still ok (ie. can access the database, does not
    have encoding issues, ...)
    3. Run the tests in t/Context.t, t/Koha/Config.t,
    t/db_dependent/Koha/Database.t, t/timezones.t
    
    Signed-off-by: Martin Renvoize's avatarMartin Renvoize <martin.renvoize@ptfs-europe.com>
    
    Signed-off-by: default avatarNick Clemens <nick@bywatersolutions.com>
    
    Signed-off-by: default avatarKyle M Hall <kyle@bywatersolutions.com>
    
    Signed-off-by: default avatarJonathan Druart <jonathan.druart@bugs.koha-community.org>
    4605304f