Unicorn does not use the correct before_fork/after_fork code
Our Unicorn setup currently does not include any before_fork/after_fork code for re-establishing ActiveRecord connections, despite the use of preload_app true
. Omnibus allows you to set these variables, but they seem to be ignored. That is, they're stored in /etc/gitlab/gitlab.rb
but not applied to the Unicorn configuration file:
$ sudo cat /etc/gitlab/gitlab.rb | grep -i fork # Yes, I still use sudo (also kinda have to due to permissions)
unicorn['before_fork'] = "defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!\n"
unicorn['after_fork'] = "defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection\n"
$ sudo cat /var/opt/gitlab/gitlab-rails/etc/unicorn.rb
# This file is managed by gitlab-ctl. Manual changes will be
# erased! To change the contents below, edit /etc/gitlab/gitlab.rb
# and run `sudo gitlab-ctl reconfigure`.
# What ports/sockets to listen on, and what options for them.
listen "127.0.0.1:8080", :tcp_nopush => true
listen "/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket", :backlog => 1024
working_directory '/var/opt/gitlab/gitlab-rails/working'
# What the timeout for killing busy workers is, in seconds
timeout 60
# Whether the app should be pre-loaded
preload_app true
# How many worker processes
worker_processes 25
# What to do before we fork a worker
before_fork do |server, worker|
old_pid = "#{server.config[:pid]}.oldbin"
if old_pid != server.pid
begin
sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
Process.kill(sig, File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
end
end
end
# Where to drop a pidfile
pid '/opt/gitlab/var/unicorn/unicorn.pid'
# Where stderr gets logged
stderr_path '/var/log/gitlab/unicorn/unicorn_stderr.log'
# Where stdout gets logged
stdout_path '/var/log/gitlab/unicorn/unicorn_stdout.log'
# Min memory size (RSS) per worker
ENV['GITLAB_UNICORN_MEMORY_MIN'] = (500000000).to_s
# Max memory size (RSS) per worker
ENV['GITLAB_UNICORN_MEMORY_MAX'] = (734003200).to_s
Editing the unicorn.rb file manually and then running gitlab-ctl reconfigure
results in the file being re-generated, but again without the forking settings.
cc @marin