Skip to content

Memoize ability_map, conditions, global_actions, delegations

George Koltsov requested to merge georgekoltsov/memoisze-policy-conditions-ability-map-delegations into main

Description

This MR memoizes:

  • DeclarativePolicy::Base.ability_map
  • DeclarativePolicy::Base.conditions
  • DeclarativePolicy::Base.global_actions
  • DeclarativePolicy::Base.delegations

In gitlab-org/gitlab#420623 (closed) after adding an ability to filter out notes the Exporter User is not permitted to read customers started reporting that their projects can not longer be exported & sidekiq node runs out of RAM during export. We filter out notes by iterating them with note.readable_by?(user). If enough notes are present, it causes memory usage to skyrocket.

This should also benefit notes rendering in the UI (and all other policy checks).

Example for 500 notes:

Before 800 Mb memory allocated:

require 'memory_profiler'

MemoryProfiler.report { Note.last(500).each { |note| note.readable_by?(User.first) }  }.pretty_print(detailed_report: true)

Total allocated: 827235195 bytes (4897665 objects)
Total retained:  3871264 bytes (20684 objects)

allocated memory by gem
-----------------------------------
 287181225  declarative_policy-1.1.0
 145281644  other
 133107381  activesupport-7.0.6
 109408432  activerecord-7.0.6


allocated memory by location
-----------------------------------
 181285568  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/declarative_policy-1.1.0/lib/declarative_policy/base.rb:77
 137862000  <internal:marshal>:35
  86137016  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/declarative_policy-1.1.0/lib/declarative_policy/base.rb:20
  57233743  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/marginalia-1.11.1/lib/marginalia/comment.rb:115
  55162000  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/activesupport-7.0.6/lib/active_support/cache.rb:1013

After 500 Mb memory allocated:

require 'memory_profiler'

MemoryProfiler.report { Note.last(500).each { |note| note.readable_by?(User.first) }  }.pretty_print(detailed_report: true)

Total allocated: 540877937 bytes (4600274 objects)
Total retained:  192823 bytes (853 objects)

allocated memory by gem
-----------------------------------
 145376235  other
 129580961  activesupport-7.0.6
 107732135  activerecord-7.0.6
 ...

allocated memory by location
-----------------------------------
 137997693  <internal:marshal>:35
  57103318  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/marginalia-1.11.1/lib/marginalia/comment.rb:115
  55217160  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/activesupport-7.0.6/lib/active_support/cache.rb:1013
  46574912  /Users/georgekoltsov/.asdf/installs/ruby/3.1.4/lib/ruby/gems/3.1.0/gems/activesupport-7.0.6/lib/active_support/core_ext/enumerable.rb:78

A rough 37% reduction in memory allocation.

Related Issues

See: #31 (closed)

Suggested version bump

  • Major (backwards incompatible changes)
  • Minor (backwards compatible changes)
  • Patch (API compatible changes)

Checklist

  • Tests have been added or updated to cover any changes in behavior
  • This does not change the API to consume this library, or a suggested version bump has been provided
  • The CHANGELOG.md has been updated
  • No new runtime dependencies have been introduced
Edited by Peter Leitzen

Merge request reports