Enumerator#next patch allocates too many objects for enumerations inside Ruby extensions
!88882 (merged) introduced a monkey patch for Enumeration#next
to improve stack traces for Gitaly calls in https://bugs.ruby-lang.org/issues/16829.
This patch relies on checking that gitlab_patch_backtrace_marker
is present in the stack trace. However, for a Rust extension like prometheus-client-mmap
, the Ruby interpreter omits the details inside the extension. For example, in omnibus-gitlab#8373 (comment 1748497473), we see the StopIterator
is handled with this backtrace:
=== EnumeratorNextPatch exception: iteration reached an end
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/enumerator_next_patch.rb:9:in `block (2 levels) in <module:EnumeratorNextPatch>'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_dict.rb:42:in `fetch_entry'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_dict.rb:42:in `read_value'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_value.rb:146:in `read_value'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_value.rb:127:in `unsafe_initiali
ze_file'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_value.rb:107:in `block in initia
lize_file'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_value.rb:106:in `synchronize'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_value.rb:106:in `initialize_file
'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/mmaped_value.rb:26:in `initialize'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/uses_value_type.rb:12:in `new'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/uses_value_type.rb:12:in `value_object'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/counter.rb:23:in `default'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/metric.rb:21:in `block in initialize'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/prometheus-client-mmap-1.1.0-x86_64-linux/lib/prometheus/client/metric.rb:37:in `get'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sli.rb:52:in `block in initialize_counters'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sli.rb:50:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sli.rb:50:in `initialize_counters'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sli.rb:20:in `block in initialize_sli'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sli.rb:16:in `synchronize'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sli.rb:16:in `initialize_sli'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/global_search_slis.rb:18:in `initialize_slis!'
This results in a large number of memory allocations when the extension iterates through the WeakMap
. As omnibus-gitlab#8373 (comment 1750521259) shows, we see a large number of backtraces being created as a result:
StopIterator
is a normal exception handled by the Ruby interpreter when the enumerator is at the end. Appending the stack trace is pretty wasteful.
Since !88882 (merged) was introduced to improve debugging for Gitaly/gRPC calls, I propose we restrict this code only to run in that context.