Sidekiq segfaults on arm64
The following discussion from !4535 (merged) should be addressed:
-
@twk3 started a discussion: (+4 comments) I'm getting ruby segfaults in both sidekiq and puma:
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71: [BUG] Segmentation fault at 0x00000000000000dc ruby 2.6.6p146 (2020-03-31 revision 67876) [aarch64-linux] -- Control frame information ----------------------------------------------- c:0063 p:---- s:0355 e:000354 CFUNC :eval c:0062 p:0021 s:0350 e:000349 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71 [FINISH] c:0061 p:---- s:0347 e:000346 CFUNC :catch c:0060 p:0014 s:0342 e:000341 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:70 c:0059 p:0030 s:0336 e:000335 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:97 [FINISH] c:0058 p:---- s:0332 e:000331 CFUNC :each c:0057 p:0039 s:0328 e:000327 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:95 c:0056 p:0077 s:0320 e:000319 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:30 c:0055 p:0060 s:0310 e:000309 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source.rb:27 c:0054 p:0031 s:0301 e:000300 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source.rb:110 c:0053 p:0051 s:0297 e:000296 BLOCK /opt/gitlab/embedded/service/gitlab-rails/config/initializers/zz_metrics.rb:181 c:0052 p:0046 s:0291 e:000290 BLOCK /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:78 [FINISH] c:0051 p:---- s:0286 e:000285 CFUNC :each c:0050 p:0025 s:0282 e:000281 METHOD /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:74 c:0049 p:0030 s:0276 e:000275 METHOD /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:57 c:0048 p:0093 s:0268 e:000267 BLOCK /opt/gitlab/embedded/service/gitlab-rails/config/initializers/zz_metrics.rb:172 c:0047 p:0017 s:0263 e:000262 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:68 c:0046 p:0037 s:0260 e:000259 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:61 c:0045 p:0016 s:0253 e:000252 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:66 c:0044 p:0013 s:0245 e:000244 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:52 [FINISH] c:0043 p:---- s:0240 e:000239 CFUNC :each c:0042 p:0035 s:0236 e:000235 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:51 c:0041 p:0014 s:0230 e:000229 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/application/finisher.rb:129 [FINISH] c:0040 p:---- s:0227 e:000226 CFUNC :instance_exec c:0039 p:0015 s:0222 e:000221 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:32 c:0038 p:0019 s:0217 e:000216 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:61 c:0037 p:0019 s:0213 e:000212 BLOCK /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:228 c:0036 p:0005 s:0209 e:000208 BLOCK /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:350 c:0035 p:0111 s:0205 E:001c60 METHOD /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:431 c:0034 p:0031 s:0193 E:000ef8 BLOCK /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:349 [FINISH] c:0033 p:---- s:0189 e:000188 CFUNC :each c:0032 p:---- s:0186 e:000185 CFUNC :call c:0031 p:0033 s:0182 E:0017b0 METHOD /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:347 c:0030 p:0034 s:0174 E:001d40 METHOD /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:226 c:0029 p:0034 s:0168 e:000167 METHOD /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:205 c:0028 p:0022 s:0161 E:001ce0 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:60 c:0027 p:0023 s:0155 e:000154 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/application.rb:363 c:0026 p:0020 s:0150 e:000149 TOP /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5 [FINISH] c:0025 p:---- s:0147 e:000146 CFUNC :require c:0024 p:0008 s:0142 e:000141 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324 c:0023 p:0068 s:0139 e:000138 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:291 c:0022 p:0011 s:0132 e:000131 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324 c:0021 p:0267 s:0126 e:000125 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.9/lib/sidekiq/cli.rb:288 c:0020 p:0025 s:0122 e:000121 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.9/lib/sidekiq/cli.rb:46 c:0019 p:0036 s:0112 e:000111 TOP /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.9/bin/sidekiq:12 [FINISH] c:0018 p:---- s:0107 e:000106 CFUNC :load c:0017 p:0132 s:0102 e:000101 TOP /opt/gitlab/embedded/bin/sidekiq:23 [FINISH] c:0016 p:---- s:0097 e:000096 CFUNC :load c:0015 p:0129 s:0092 e:000091 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74 c:0014 p:0075 s:0083 e:000082 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28 c:0013 p:0026 s:0078 e:000077 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463 c:0012 p:0064 s:0073 e:000072 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27 c:0011 p:0047 s:0065 e:000064 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126 c:0010 p:0261 s:0058 e:000057 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387 c:0009 p:0009 s:0045 e:000044 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27 c:0008 p:0064 s:0040 e:000039 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466 c:0007 p:0009 s:0033 e:000032 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18 c:0006 p:0076 s:0027 e:000026 BLOCK /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:30 c:0005 p:0002 s:0021 e:000020 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124 c:0004 p:0048 s:0016 E:001e10 TOP /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:22 [FINISH] c:0003 p:---- s:0013 e:000012 CFUNC :load c:0002 p:0132 s:0008 E:001150 EVAL /opt/gitlab/embedded/bin/bundle:23 [FINISH] c:0001 p:0000 s:0003 E:000b50 (none) [FINISH] -- Ruby level backtrace information ---------------------------------------- /opt/gitlab/embedded/bin/bundle:23:in `<main>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)>' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)>' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load' /opt/gitlab/embedded/bin/sidekiq:23:in `<top (required)>' /opt/gitlab/embedded/bin/sidekiq:23:in `load' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.9/bin/sidekiq:12:in `<top (required)>' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.9/lib/sidekiq/cli.rb:46:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.9/lib/sidekiq/cli.rb:288:in `boot_system' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324:in `require' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:291:in `load_dependency' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324:in `block in require' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324:in `require' /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/application.rb:363:in `initialize!' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:60:in `run_initializers' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:205:in `tsort_each' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:226:in `tsort_each' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:347:in `each_strongly_connected_component' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:347:in `call' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:347:in `each' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:349:in `block in each_strongly_connected_component' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:431:in `each_strongly_connected_component_from' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component' /opt/gitlab/embedded/lib/ruby/2.6.0/tsort.rb:228:in `block in tsort_each' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:61:in `block in run_initializers' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:32:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:32:in `instance_exec' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/railties-6.0.3.1/lib/rails/application/finisher.rb:129:in `block in <module:Finisher>' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:51:in `run_load_hooks' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:51:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:52:in `block in run_load_hooks' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:66:in `execute_hook' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:61:in `with_execution_control' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:68:in `block in execute_hook' /opt/gitlab/embedded/service/gitlab-rails/config/initializers/zz_metrics.rb:172:in `block in <top (required)>' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:57:in `instrument_class_hierarchy' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:74:in `instrument_methods' /opt/gitlab/embedded/service/gitlab-rails/config/initializers/zz_metrics.rb:181:in `block (2 levels) in <top (required)>' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source.rb:110:in `source' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source.rb:27:in `source_helper' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:30:in `expression_at' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:95:in `extract_first_expression' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:95:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:97:in `block in extract_first_expression' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:70:in `complete_expression?' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:70:in `catch' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71:in `block in complete_expression?' /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71:in `eval'This is on an aws
a1.xlargerunning the openSUSE Leap 15.1 ARM marketplace ami image.