ruby crashes with SIGSEGV in omnibus 10.6.0-ce.0 package on debian 9.4 due to libjemalloc.so in LD_PRELOAD
Summary
Updating gitlab-ce omnibus debian package 10.5.4-ce.0 -> 10.6.0-ce.0 gives:
Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
Relevant logs are attached below.
Manual attempts to execute failed db migration action gives SIGSEGV :
>/opt/gitlab/bin/gitlab-rake gitlab:db:configure
Segmentation faultFurther investigation shows that:
- ruby crashes on early stages after
clone()syscall(see strace output and gdb backtrace below) - crash related to libjemalloc.so loaded by
/opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD
Removing /opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD fixes everything, so manually running gitlab-rake gitlab:db:configure afterwards gives usable installation.
I am not sure if the problem is with debian's libc6 2.24-11+deb9u3 or with omnibus package's libjemalloc.so/ruby or with something else, but some combination of those definitely fails to work on my installation.
Steps to reproduce
> sudo apt-get install gitlab-ce on Debian 9.4 x64 with libc6 version: 2.24-11+deb9u3
What is the current bug behavior?
> gitlab-rails
Segmentation faultWhat is the expected correct behavior?
Working ruby installation is expected.
Relevant logs
Expand for /var/log/gitlab/reconfigure/1521734238.log
# Logfile created on 2018-03-22 16:57:18 +0100 by logger.rb/56438 [2018-03-22T16:57:18+01:00] INFO: Started chef-zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded One version per cookbook[2018-03-22T16:57:18+01:00] INFO: *** Chef 13.6.4 *** [2018-03-22T16:57:18+01:00] INFO: Platform: x86_64-linux [2018-03-22T16:57:18+01:00] INFO: Chef-client pid: 22995 [2018-03-22T16:57:18+01:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping... [2018-03-22T16:57:19+01:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options [2018-03-22T16:57:19+01:00] INFO: Run List is [recipe[gitlab]] [2018-03-22T16:57:19+01:00] INFO: Run List expands to [gitlab] [2018-03-22T16:57:19+01:00] INFO: Starting Chef Run for poligon-dev [2018-03-22T16:57:19+01:00] INFO: Running start handlers [2018-03-22T16:57:19+01:00] INFO: Start handlers complete. [2018-03-22T16:57:20+01:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, postgresql@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.0.0, gitaly@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@0.14.2, acme@3.1.0, compat_resource@12.19.0] [2018-03-22T16:57:22+01:00] WARN: Selected systemd because systemctl shows .mount units [2018-03-22T16:57:23+01:00] INFO: execute[/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions] ran successfully [2018-03-22T16:57:23+01:00] INFO: template[/var/opt/gitlab/gitlab-rails/etc/gitlab.yml] backed up to /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-rails/etc/gitlab.yml.chef-20180322165723.829479 [2018-03-22T16:57:23+01:00] INFO: template[/var/opt/gitlab/gitlab-rails/etc/gitlab.yml] removed backup at /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-rails/etc/gitlab.yml.chef-20180206233236.354804 [2018-03-22T16:57:23+01:00] INFO: template[/var/opt/gitlab/gitlab-rails/etc/gitlab.yml] updated file contents /var/opt/gitlab/gitlab-rails/etc/gitlab.yml [2018-03-22T16:57:23+01:00] INFO: file[/opt/gitlab/embedded/service/gitlab-rails/db/schema.rb] owner changed to 998 [2018-03-22T16:57:23+01:00] INFO: remote_file[/var/opt/gitlab/gitlab-rails/VERSION] backed up to /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-rails/VERSION.chef-20180322165723.912794 [2018-03-22T16:57:23+01:00] INFO: remote_file[/var/opt/gitlab/gitlab-rails/VERSION] removed backup at /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-rails/VERSION.chef-20180209123327.753942 [2018-03-22T16:57:23+01:00] INFO: remote_file[/var/opt/gitlab/gitlab-rails/VERSION] updated file contents /var/opt/gitlab/gitlab-rails/VERSION [2018-03-22T16:57:23+01:00] INFO: remote_file[/var/opt/gitlab/gitlab-rails/REVISION] backed up to /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-rails/REVISION.chef-20180322165723.916943 [2018-03-22T16:57:23+01:00] INFO: remote_file[/var/opt/gitlab/gitlab-rails/REVISION] removed backup at /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-rails/REVISION.chef-20180209123327.757641 [2018-03-22T16:57:23+01:00] INFO: remote_file[/var/opt/gitlab/gitlab-rails/REVISION] updated file contents /var/opt/gitlab/gitlab-rails/REVISION [2018-03-22T16:57:23+01:00] INFO: execute[chown -R root:root /opt/gitlab/embedded/service/gitlab-rails/public] ran successfully [2018-03-22T16:57:24+01:00] INFO: ruby_block[Move existing certs and link to /opt/gitlab/embedded/ssl/certs] called [2018-03-22T16:57:24+01:00] INFO: template[/var/opt/gitlab/redis/redis.conf] backed up to /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/redis/redis.conf.chef-20180322165724.047360 [2018-03-22T16:57:24+01:00] INFO: template[/var/opt/gitlab/redis/redis.conf] updated file contents /var/opt/gitlab/redis/redis.conf [2018-03-22T16:57:24+01:00] INFO: template[/var/opt/gitlab/redis/redis.conf] sending restart action to service[redis] (immediate) [2018-03-22T16:57:24+01:00] INFO: service[redis] restarted [2018-03-22T16:57:25+01:00] INFO: Running queued delayed notifications before re-raising exception [2018-03-22T16:57:25+01:00] INFO: templatesymlink[Create a gitlab.yml and create a symlink to Rails root] sending run action to execute[clear the gitlab-rails cache] (delayed) [2018-03-22T16:57:25+01:00] ERROR: Running exception handlers [2018-03-22T16:57:25+01:00] ERROR: Exception handlers complete [2018-03-22T16:57:25+01:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/chef-stacktrace.out [2018-03-22T16:57:25+01:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report [2018-03-22T16:57:25+01:00] FATAL: Chef::Exceptions::MultipleFailures: Multiple failures occurred:
- Mixlib::ShellOut::ShellCommandFailed occurred in chef run: bash[migrate gitlab-rails database] (gitlab::database_migrations line 49) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '139' ---- Begin output of "bash" "/tmp/chef-script20180322-22995-1o731o0" ---- STDOUT: STDERR: ---- End output of "bash" "/tmp/chef-script20180322-22995-1o731o0" ---- Ran "bash" "/tmp/chef-script20180322-22995-1o731o0" returned 139
- Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: execute[clear the gitlab-rails cache] (gitlab::gitlab-rails line 390) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '' ---- Begin output of /opt/gitlab/bin/gitlab-rake cache:clear ---- STDOUT: STDERR: ---- End output of /opt/gitlab/bin/gitlab-rake cache:clear ---- Ran /opt/gitlab/bin/gitlab-rake cache:clear returned
Expand for /opt/gitlab/embedded/cookbooks/cache/chef-stacktrace.out
Generated at 2018-03-22 20:31:42 +0300 Chef::Exceptions::MultipleFailures: Chef::Exceptions::MultipleFailures /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:293:in `invalid!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:280:in `error!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/mixin/shell_out.rb:130:in `shell_out_with_systems_locale!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/execute.rb:58:in `block in action_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/mixin/why_run.rb:52:in `add_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider.rb:202:in `converge_by' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/execute.rb:56:in `action_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/script.rb:64:in `action_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider.rb:171:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource.rb:591:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:70:in `run_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:in `block (2 levels) in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:in `block in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource' /opt/gitlab/embedded/lib/ruby/2.3.0/forwardable.rb:204:in `execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:97:in `converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:718:in `block in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:713:in `catch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:713:in `converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:752:in `converge_and_save' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:286:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:273:in `run_with_graceful_exit_option' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:249:in `block in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/local_mode.rb:44:in `with_server_connectivity' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:232:in `run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application/client.rb:434:in `run_application' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:59:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/bin/chef-client:26:in `' /opt/gitlab/embedded/bin/chef-client:23:in `load' /opt/gitlab/embedded/bin/chef-client:23:in `'Caused by Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (gitlab::database_migrations line 49) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '139' ---- Begin output of "bash" "/tmp/chef-script20180322-23560-1geikm6" ---- STDOUT: STDERR: ---- End output of "bash" "/tmp/chef-script20180322-23560-1geikm6" ---- Ran "bash" "/tmp/chef-script20180322-23560-1geikm6" returned 139 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:293:in
invalid!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:280:inerror!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/mixin/shell_out.rb:130:inshell_out_with_systems_locale!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/execute.rb:58:inblock in action_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/mixin/why_run.rb:52:inadd_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider.rb:202:inconverge_by' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/execute.rb:56:inaction_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/script.rb:64:inaction_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider.rb:171:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource.rb:591:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:70:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:inblock (2 levels) in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:ineach' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:inblock in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/resource_list.rb:94:inblock in execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:114:incall_iterator_block' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:85:instep' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:103:initerate' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:55:ineach_with_index' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/resource_list.rb:92:inexecute_each_resource' /opt/gitlab/embedded/lib/ruby/2.3.0/forwardable.rb:204:inexecute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:97:inconverge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:718:inblock in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:713:incatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:713:inconverge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:752:inconverge_and_save' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:286:inrun' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:273:inrun_with_graceful_exit_option' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:249:inblock in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/local_mode.rb:44:inwith_server_connectivity' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:232:inrun_chef_client' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application/client.rb:434:inrun_application' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:59:inrun' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/bin/chef-client:26:in<top (required)>' /opt/gitlab/embedded/bin/chef-client:23:inload' /opt/gitlab/embedded/bin/chef-client:23:in'Caused by Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '139' ---- Begin output of "bash" "/tmp/chef-script20180322-23560-1geikm6" ---- STDOUT: STDERR: ---- End output of "bash" "/tmp/chef-script20180322-23560-1geikm6" ---- Ran "bash" "/tmp/chef-script20180322-23560-1geikm6" returned 139 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:293:in
invalid!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:280:inerror!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/mixin/shell_out.rb:130:inshell_out_with_systems_locale!' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/execute.rb:58:inblock in action_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/mixin/why_run.rb:52:inadd_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider.rb:202:inconverge_by' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/execute.rb:56:inaction_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider/script.rb:64:inaction_run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/provider.rb:171:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource.rb:591:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:70:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:inblock (2 levels) in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:ineach' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:98:inblock in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/resource_list.rb:94:inblock in execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:114:incall_iterator_block' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:85:instep' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:103:initerate' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/stepable_iterator.rb:55:ineach_with_index' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/resource_collection/resource_list.rb:92:inexecute_each_resource' /opt/gitlab/embedded/lib/ruby/2.3.0/forwardable.rb:204:inexecute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/runner.rb:97:inconverge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:718:inblock in converge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:713:incatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:713:inconverge' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:752:inconverge_and_save' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/client.rb:286:inrun' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:273:inrun_with_graceful_exit_option' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:249:inblock in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/local_mode.rb:44:inwith_server_connectivity' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:232:inrun_chef_client' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application/client.rb:434:inrun_application' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/lib/chef/application.rb:59:inrun' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/chef-13.6.4/bin/chef-client:26:in<top (required)>' /opt/gitlab/embedded/bin/chef-client:23:inload' /opt/gitlab/embedded/bin/chef-client:23:in'
Expand for gdb backtrace of ruby
# su git $ export BUNDLE_GEMFILE=/opt/gitlab/embedded/service/gitlab-rails/Gemfile $ export EXECJS_RUNTIME=Disabled $ export HOME=/var/opt/gitlab $ export ICU_DATA=/opt/gitlab/embedded/share/icu/current $ export LD_PRELOAD=/opt/gitlab/embedded/lib/libjemalloc.so $ export PATH=/opt/gitlab/bin:/opt/gitlab/embedded/bin:/bin:/usr/bin $ export PYTHONPATH=/opt/gitlab/embedded/lib/python3.4/site-packages $ export RAILS_ENV=production $ export SIDEKIQ_MEMORY_KILLER_MAX_RSS=1000000 $ gdb ruby GNU gdb (Debian 7.12-6) 7.12.0.20161007-git Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ruby...done. (gdb) run Starting program: /opt/gitlab/embedded/bin/ruby [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7ffff7ff4700 (LWP 1809)]Thread 2 "ruby" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff7ff4700 (LWP 1809)] 0x00007ffff7def2ef in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7def2ef in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff722e40b in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007ffff6830acf in clone () from /lib/x86_64-linux-gnu/libc.so.6 (gdb)
Expand for strace gitlab-rails
# strace gitlab-rails
execve("/usr/bin/gitlab-rails", ["gitlab-rails"], [/* 17 vars */]) = 0
brk(NULL) = 0x55906f23c000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f98c3251000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=17452, ...}) = 0
mmap(NULL, 17452, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f98c324c000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\4\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1689360, ...}) = 0
mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f98c2c92000
mprotect(0x7f98c2e27000, 2097152, PROT_NONE) = 0
mmap(0x7f98c3027000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x195000) = 0x7f98c3027000
mmap(0x7f98c302d000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f98c302d000
close(3) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f98c324a000
arch_prctl(ARCH_SET_FS, 0x7f98c324a700) = 0
mprotect(0x7f98c3027000, 16384, PROT_READ) = 0
mprotect(0x55906d753000, 8192, PROT_READ) = 0
mprotect(0x7f98c3254000, 4096, PROT_READ) = 0
munmap(0x7f98c324c000, 17452) = 0
getpid() = 1423
rt_sigaction(SIGCHLD, {sa_handler=0x55906d549ef0, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7f98c2cc5060}, NULL, 8) = 0
geteuid() = 0
brk(NULL) = 0x55906f23c000
brk(0x55906f25d000) = 0x55906f25d000
getppid() = 1421
stat("/root", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
stat(".", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
open("/usr/bin/gitlab-rails", O_RDONLY) = 3
fcntl(3, F_DUPFD, 10) = 10
close(3) = 0
fcntl(10, F_SETFD, FD_CLOEXEC) = 0
rt_sigaction(SIGINT, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x55906d549ef0, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7f98c2cc5060}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGQUIT, {sa_handler=SIG_DFL, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7f98c2cc5060}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGTERM, {sa_handler=SIG_DFL, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7f98c2cc5060}, NULL, 8) = 0
read(10, "#!/bin/sh\n\n# Unset ENV variables"..., 8192) = 1269
stat("/opt/gitlab/etc/gitlab-rails/gitlab-rails-rc", {st_mode=S_IFREG|0644, st_size=18, ...}) = 0
open("/opt/gitlab/etc/gitlab-rails/gitlab-rails-rc", O_RDONLY) = 3
fcntl(3, F_DUPFD, 10) = 11
close(3) = 0
fcntl(11, F_SETFD, FD_CLOEXEC) = 0
read(11, "gitlab_user='git'\n", 8192) = 18
read(11, "", 8192) = 0
close(11) = 0
chdir("/opt/gitlab/embedded/service/gitlab-rails") = 0
pipe([3, 4]) = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f98c324a9d0) = 1424
close(4) = 0
read(3, "root\n", 128) = 5
read(3, "", 128) = 0
close(3) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1424, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn({mask=[]}) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 1424
execve("/opt/gitlab/embedded/bin/chpst", ["/opt/gitlab/embedded/bin/chpst", "-e", "/opt/gitlab/etc/gitlab-rails/env", "-u", "git", "-U", "git", "/opt/gitlab/embedded/bin/bundle", "exec", "rails"], [/* 18 vars */]) = 0
brk(NULL) = 0x5610f81d1000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f28f2c4c000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib/tls/x86_64", 0x7ffc848adb80) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib/tls", 0x7ffc848adb80) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib/x86_64", 0x7ffc848adb80) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib", {st_mode=S_IFDIR|0755, st_size=36864, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=17452, ...}) = 0
mmap(NULL, 17452, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f28f2c47000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\4\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1689360, ...}) = 0
mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f28f268d000
mprotect(0x7f28f2822000, 2097152, PROT_NONE) = 0
mmap(0x7f28f2a22000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x195000) = 0x7f28f2a22000
mmap(0x7f28f2a28000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f28f2a28000
close(3) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f28f2c45000
arch_prctl(ARCH_SET_FS, 0x7f28f2c45700) = 0
mprotect(0x7f28f2a22000, 16384, PROT_READ) = 0
mprotect(0x5610f75bc000, 4096, PROT_READ) = 0
mprotect(0x7f28f2c4f000, 4096, PROT_READ) = 0
munmap(0x7f28f2c47000, 17452) = 0
open(".", O_RDONLY|O_NONBLOCK) = 3
chdir("/opt/gitlab/etc/gitlab-rails/env") = 0
open(".", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
brk(NULL) = 0x5610f81d1000
brk(0x5610f81fa000) = 0x5610f81fa000
getdents(4, /* 11 entries */, 32768) = 360
open("PYTHONPATH", O_RDONLY|O_NONBLOCK) = 5
read(5, "/opt/gitlab/embedded/lib/python3"..., 256) = 48
read(5, "", 256) = 0
close(5) = 0
open("EXECJS_RUNTIME", O_RDONLY|O_NONBLOCK) = 5
read(5, "Disabled", 256) = 8
read(5, "", 256) = 0
close(5) = 0
open("ICU_DATA", O_RDONLY|O_NONBLOCK) = 5
read(5, "/opt/gitlab/embedded/share/icu/c"..., 256) = 38
read(5, "", 256) = 0
close(5) = 0
open("PATH", O_RDONLY|O_NONBLOCK) = 5
read(5, "/opt/gitlab/bin:/opt/gitlab/embe"..., 256) = 54
read(5, "", 256) = 0
close(5) = 0
open("RAILS_ENV", O_RDONLY|O_NONBLOCK) = 5
read(5, "production", 256) = 10
read(5, "", 256) = 0
close(5) = 0
open("BUNDLE_GEMFILE", O_RDONLY|O_NONBLOCK) = 5
read(5, "/opt/gitlab/embedded/service/git"..., 256) = 49
read(5, "", 256) = 0
close(5) = 0
open("SIDEKIQ_MEMORY_KILLER_MAX_RSS", O_RDONLY|O_NONBLOCK) = 5
read(5, "1000000", 256) = 7
read(5, "", 256) = 0
close(5) = 0
open("HOME", O_RDONLY|O_NONBLOCK) = 5
read(5, "/var/opt/gitlab", 256) = 15
read(5, "", 256) = 0
close(5) = 0
open("LD_PRELOAD", O_RDONLY|O_NONBLOCK) = 5
read(5, "/opt/gitlab/embedded/lib/libjema"..., 256) = 39
read(5, "", 256) = 0
close(5) = 0
getdents(4, /* 0 entries */, 32768) = 0
brk(0x5610f81f2000) = 0x5610f81f2000
close(4) = 0
fchdir(3) = 0
close(3) = 0
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=497, ...}) = 0
read(3, "# /etc/nsswitch.conf\n#\n# Example"..., 4096) = 497
read(3, "", 4096) = 0
close(3) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=17452, ...}) = 0
mmap(NULL, 17452, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f28f2c47000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_compat.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\22\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31616, ...}) = 0
mmap(NULL, 2126944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f28f2485000
mprotect(0x7f28f248c000, 2093056, PROT_NONE) = 0
mmap(0x7f28f268b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f28f268b000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnsl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320?\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=89064, ...}) = 0
mmap(NULL, 2194008, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f28f226d000
mprotect(0x7f28f2281000, 2097152, PROT_NONE) = 0
mmap(0x7f28f2481000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14000) = 0x7f28f2481000
mmap(0x7f28f2483000, 6744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f28f2483000
close(3) = 0
mprotect(0x7f28f2481000, 4096, PROT_READ) = 0
mprotect(0x7f28f268b000, 4096, PROT_READ) = 0
munmap(0x7f28f2c47000, 17452) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=17452, ...}) = 0
mmap(NULL, 17452, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f28f2c47000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_nis.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340 \0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=47688, ...}) = 0
mmap(NULL, 2143656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f28f2061000
mprotect(0x7f28f206c000, 2093056, PROT_NONE) = 0
mmap(0x7f28f226b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x7f28f226b000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320!\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=47632, ...}) = 0
mmap(NULL, 2168600, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f28f1e4f000
mprotect(0x7f28f1e59000, 2097152, PROT_NONE) = 0
mmap(0x7f28f2059000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x7f28f2059000
mmap(0x7f28f205b000, 22296, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f28f205b000
close(3) = 0
mprotect(0x7f28f2059000, 4096, PROT_READ) = 0
mprotect(0x7f28f226b000, 4096, PROT_READ) = 0
munmap(0x7f28f2c47000, 17452) = 0
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
lseek(3, 0, SEEK_CUR) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=1605, ...}) = 0
mmap(NULL, 1605, PROT_READ, MAP_SHARED, 3, 0) = 0x7f28f2c4b000
lseek(3, 1605, SEEK_SET) = 1605
munmap(0x7f28f2c4b000, 1605) = 0
close(3) = 0
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
lseek(3, 0, SEEK_CUR) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=1605, ...}) = 0
mmap(NULL, 1605, PROT_READ, MAP_SHARED, 3, 0) = 0x7f28f2c4b000
lseek(3, 1605, SEEK_SET) = 1605
munmap(0x7f28f2c4b000, 1605) = 0
close(3) = 0
setgroups(1, [998]) = 0
setgid(998) = 0
setuid(998) = 0
execve("/opt/gitlab/embedded/bin/bundle", ["/opt/gitlab/embedded/bin/bundle", "exec", "rails"], [/* 27 vars */]) = 0
brk(NULL) = 0x55ac2520b000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc6f64f3000
open("/opt/gitlab/embedded/lib/libjemalloc.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P^\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=4379744, ...}) = 0
mmap(NULL, 4879872, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f5e2b000
mprotect(0x7fc6f5eae000, 2093056, PROT_NONE) = 0
mmap(0x7fc6f60ad000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x82000) = 0x7fc6f60ad000
mmap(0x7fc6f60b3000, 2225664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc6f60b3000
close(3) = 0
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/tls/x86_64/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib/tls/x86_64", 0x7ffdf4d66310) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/tls/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib/tls", 0x7ffdf4d66310) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/x86_64/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/gitlab/embedded/lib/x86_64", 0x7ffdf4d66310) = -1 ENOENT (No such file or directory)
open("/opt/gitlab/embedded/lib/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\326\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=13792048, ...}) = 0
mmap(NULL, 5161088, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f593e000
mprotect(0x7fc6f5c12000, 2093056, PROT_NONE) = 0
mmap(0x7fc6f5e11000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2d3000) = 0x7fc6f5e11000
mmap(0x7fc6f5e1a000, 65664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc6f5e1a000
close(3) = 0
open("/opt/gitlab/embedded/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=17452, ...}) = 0
mmap(NULL, 17452, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc6f64ee000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0Pa\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=135440, ...}) = 0
mmap(NULL, 2212936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f5721000
mprotect(0x7fc6f5739000, 2093056, PROT_NONE) = 0
mmap(0x7fc6f5938000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fc6f5938000
mmap(0x7fc6f593a000, 13384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc6f593a000
close(3) = 0
open("/opt/gitlab/embedded/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\r\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14640, ...}) = 0
mmap(NULL, 2109680, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f551d000
mprotect(0x7fc6f5520000, 2093056, PROT_NONE) = 0
mmap(0x7fc6f571f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fc6f571f000
close(3) = 0
open("/opt/gitlab/embedded/lib/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\v\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=39256, ...}) = 0
mmap(NULL, 2322912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f52e5000
mprotect(0x7fc6f52ed000, 2097152, PROT_NONE) = 0
mmap(0x7fc6f54ed000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x8000) = 0x7fc6f54ed000
mmap(0x7fc6f54ef000, 184800, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc6f54ef000
close(3) = 0
open("/opt/gitlab/embedded/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200V\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1063328, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc6f64ec000
mmap(NULL, 3158248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f4fe1000
mprotect(0x7fc6f50e4000, 2093056, PROT_NONE) = 0
mmap(0x7fc6f52e3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x102000) = 0x7fc6f52e3000
close(3) = 0
open("/opt/gitlab/embedded/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\4\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1689360, ...}) = 0
mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f4c42000
mprotect(0x7fc6f4dd7000, 2097152, PROT_NONE) = 0
mmap(0x7fc6f4fd7000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x195000) = 0x7fc6f4fd7000
mmap(0x7fc6f4fdd000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc6f4fdd000
close(3) = 0
open("/opt/gitlab/embedded/lib/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\267\10\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1566168, ...}) = 0
mmap(NULL, 3674720, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f48c0000
mprotect(0x7fc6f4a32000, 2097152, PROT_NONE) = 0
mmap(0x7fc6f4c32000, 49152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x172000) = 0x7fc6f4c32000
mmap(0x7fc6f4c3e000, 12896, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc6f4c3e000
close(3) = 0
open("/opt/gitlab/embedded/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220*\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=92584, ...}) = 0
mmap(NULL, 2188336, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc6f46a9000
mprotect(0x7fc6f46bf000, 2093056, PROT_NONE) = 0
mmap(0x7fc6f48be000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7fc6f48be000
close(3) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc6f64ea000
mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc6f64e6000
arch_prctl(ARCH_SET_FS, 0x7fc6f64e7f00) = 0
mprotect(0x7fc6f4fd7000, 16384, PROT_READ) = 0
mprotect(0x7fc6f48be000, 4096, PROT_READ) = 0
mprotect(0x7fc6f52e3000, 4096, PROT_READ) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc6f64e4000
mprotect(0x7fc6f4c32000, 40960, PROT_READ) = 0
mprotect(0x7fc6f54ed000, 4096, PROT_READ) = 0
mprotect(0x7fc6f571f000, 4096, PROT_READ) = 0
mprotect(0x7fc6f5938000, 4096, PROT_READ) = 0
mprotect(0x7fc6f5e11000, 24576, PROT_READ) = 0
mprotect(0x7fc6f60ad000, 20480, PROT_READ) = 0
mprotect(0x55ac23d0f000, 4096, PROT_READ) = 0
mprotect(0x7fc6f64f6000, 4096, PROT_READ) = 0
munmap(0x7fc6f64ee000, 17452) = 0
set_tid_address(0x7fc6f64e81d0) = 1423
set_robust_list(0x7fc6f64e81e0, 24) = 0
rt_sigaction(SIGRTMIN, {sa_handler=0x7fc6f5726bd0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {sa_handler=0x7fc6f5726c60, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
readlink("/etc/malloc.conf", 0x7ffdf4d65c30, 4096) = -1 ENOENT (No such file or directory)
open("/proc/sys/vm/overcommit_memory", O_RDONLY|O_CLOEXEC) = 3
read(3, "0", 1) = 1
close(3) = 0
mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fc6f44a9000
brk(NULL) = 0x55ac2520b000
mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fc6f42a9000
mmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fc6f3ea9000
sched_getaffinity(1423, 128, [0, 1]) = 64
futex(0x7fc6f57200a8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7fc6f48bf1a0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2995
read(3, "", 4096) = 0
close(3) = 0
open("/usr/lib/locale/en_US.UTF-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en_US.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=328180, ...}) = 0
mmap(NULL, 328180, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc6f6493000
close(3) = 0
open("/usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=26258, ...}) = 0
mmap(NULL, 26258, PROT_READ, MAP_SHARED, 3, 0) = 0x7fc6f648c000
close(3) = 0
futex(0x7fc6f4fdc8a8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
fstat(0, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 0), ...}) = 0
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 0), ...}) = 0
fstat(2, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 0), ...}) = 0
open("/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "55ac23b0f000-55ac23b10000 r-xp 0"..., 1024) = 1024
read(3, ".6.0.22\n7fc6f4c32000-7fc6f4c3c00"..., 1024) = 1024
read(3, "0102000 08:01 1677 "..., 1024) = 1024
read(3, "6f5721000 rw-p 00003000 08:01 16"..., 1024) = 1024
read(3, "lab/embedded/lib/libruby.so.2.3."..., 1024) = 1024
read(3, "00 0 \n7fc6f64f6000-7fc6f64f7000 "..., 1024) = 592
close(3) = 0
sched_getaffinity(1423, 32, [0, 1]) = 32
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
rt_sigaction(SIGVTALRM, {sa_handler=0x7fc6f5b32880, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
mmap(NULL, 2621440, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fc6f3c29000
clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {tv_sec=0, tv_nsec=10958634}) = 0
getrandom("?\251$\365\360\203\272\334U\373v\0107W)\300", 16, 0) = 16
getpid() = 1423
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
getrandom("\215`\366z\177\210\376\260\331@\265\333\206\213\5\355", 16, 0) = 16
getpid() = 1423
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], NULL, 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGHUP, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGQUIT, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGTERM, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGALRM, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGUSR1, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGUSR2, {sa_handler=0x7fc6f5a98f80, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x7fc6f5a98d40, sa_mask=[], sa_flags=SA_RESTORER|SA_STACK|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGILL, {sa_handler=0x7fc6f5a98c50, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
sigaltstack({ss_sp=0x7fc6f3c51640, ss_flags=0, ss_size=16384}, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x7fc6f5a98d00, sa_mask=[], sa_flags=SA_RESTORER|SA_STACK|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=0x7fc6f5a98ae0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGSYS, {sa_handler=0x7fc6f5a98ae0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGCHLD, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7fc6f57320c0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
pipe2([3, 4], O_CLOEXEC) = 0
fcntl(3, F_GETFL) = 0 (flags O_RDONLY)
fcntl(3, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
fcntl(4, F_GETFL) = 0x1 (flags O_WRONLY)
fcntl(4, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
pipe2([5, 6], O_CLOEXEC) = 0
fcntl(5, F_GETFL) = 0 (flags O_RDONLY)
fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
fcntl(6, F_GETFL) = 0x1 (flags O_WRONLY)
fcntl(6, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fc6f64ef000
mprotect(0x7fc6f64ef000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fc6f64f07f0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc6f64f29d0, tls=0x7fc6f64f2700, child_tidptr=0x7fc6f64f29d0) = 1425
+++ killed by SIGSEGV +++
Segmentation fault
Results of GitLab environment info
Suggested output of gitlab-rake gitlab:env:info is obviously Segmentation fault as well.
If libjemalloc.so is removed from LD_PRELOAD, however, the output is following
Expand for output related to GitLab environment info
System information System: Debian 9.4 Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknownGitLab information Version: 10.6.0 Revision: 8f82e53 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://some-host HTTP Clone URL: https://some-host/some-group/some-project.git SSH Clone URL: git@some-host:some-group/some-project.git Using LDAP: no Using Omniauth: no
GitLab Shell Version: 6.0.4 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Suggested output of gitlab-rake gitlab:check SANITIZE=true is obviously Segmentation fault as well.
If libjemalloc.so is removed from LD_PRELOAD, however, the output is following
Expand for output related to the GitLab application check
Checking GitLab Shell ...GitLab Shell version >= 6.0.4 ? ... OK (6.0.4) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 1/2 ... repository is empty 1/3 ... repository is empty 1/4 ... ok 1/5 ... ok 1/6 ... ok 1/7 ... ok 1/8 ... ok 1/9 ... ok 1/10 ... ok 1/14 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
IMAP server credentials are correct? ... yes Init.d configured correctly? ... skipped MailRoom running? ... skipped
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 1/2 ... yes 1/3 ... yes 1/4 ... yes 1/5 ... yes 1/6 ... yes 1/7 ... yes 1/8 ... yes 1/9 ... yes 1/10 ... yes 1/14 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 8
Checking GitLab ... Finished
Possible fixes
Something like
mv /opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD /opt/gitlab/etc/gitlab-rails/env/LD_DONT_PRELOADfixes the problem with crashing ruby. So manual running of db migration
/opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>&1 | tee /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log
succeeds and gitlab is finally running after restart.
I hope this can be of help.