uninitialized constant ActiveRecord::Observer (NameError)
Summary Unicorn fails to start, with an error in stderr regarding ActiveRecord::Observer.
Steps to reproduce:
Still investigating; However, suspicions are that restoring an omnibus database in non-omnibus Gitlab produces this behavior. Other alternatives:
- Omnibus screwing up the environment post-uninstall.
- Debian Jessie not being compatible.
- Postgres 9.4 fun.
Expected Behavior
Unicorn starts successfully without error.
Observed Behavior
Unicorn fails to start with the following output in unicorn.stderr.log:
I, [2014-09-23T05:25:44.571011 #16963] INFO -- : Refreshing Gem list
[deprecated] I18n.enforce_available_locales will default to true in the future. If you really want to skip validation of your locale you can set I18n.enforce_available_locales = false to avoid this message.
/home/git/gitlab/app/observers/base_observer.rb:1:in `<top (required)>': uninitialized constant ActiveRecord::Observer (NameError)
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/engine.rb:468:in `block (2 levels) in eager_load!'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/engine.rb:467:in `each'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/engine.rb:467:in `block in eager_load!'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/engine.rb:465:in `each'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/engine.rb:465:in `eager_load!'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/engine.rb:346:in `eager_load!'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/application/finisher.rb:58:in `each'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/application/finisher.rb:58:in `block in <module:Finisher>'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/initializable.rb:30:in `instance_exec'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/initializable.rb:30:in `run'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/initializable.rb:55:in `block in run_initializers'
from /usr/local/lib/ruby/2.1.0/tsort.rb:226:in `block in tsort_each'
from /usr/local/lib/ruby/2.1.0/tsort.rb:348:in `block (2 levels) in each_strongly_connected_component'
from /usr/local/lib/ruby/2.1.0/tsort.rb:427:in `each_strongly_connected_component_from'
from /usr/local/lib/ruby/2.1.0/tsort.rb:347:in `block in each_strongly_connected_component'
from /usr/local/lib/ruby/2.1.0/tsort.rb:345:in `each'
from /usr/local/lib/ruby/2.1.0/tsort.rb:345:in `call'
from /usr/local/lib/ruby/2.1.0/tsort.rb:345:in `each_strongly_connected_component'
from /usr/local/lib/ruby/2.1.0/tsort.rb:224:in `tsort_each'
from /usr/local/lib/ruby/2.1.0/tsort.rb:205:in `tsort_each'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/initializable.rb:54:in `run_initializers'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/application.rb:288:in `initialize!'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/railtie.rb:194:in `public_send'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/railties-4.1.1/lib/rails/railtie.rb:194:in `method_missing'
from /home/git/gitlab/config/environment.rb:5:in `<top (required)>'
from config.ru:13:in `require'
from config.ru:13:in `block in <main>'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/rack-1.5.2/lib/rack/builder.rb:55:in `instance_eval'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/rack-1.5.2/lib/rack/builder.rb:55:in `initialize'
from config.ru:1:in `new'
from config.ru:1:in `<main>'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/lib/unicorn.rb:48:in `eval'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/lib/unicorn.rb:48:in `block in builder'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/bin/unicorn_rails:139:in `call'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/bin/unicorn_rails:139:in `block in rails_builder'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:722:in `call'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:722:in `build_app!'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:140:in `start'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/gems/unicorn-4.6.3/bin/unicorn_rails:209:in `<top (required)>'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/bin/unicorn_rails:23:in `load'
from /home/git/gitlab/vendor/bundle/ruby/2.1.0/bin/unicorn_rails:23:in `<main>'
Output of checks
+ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
[deprecated] I18n.enforce_available_locales will default to true in the future. If you really want to skip validation of your locale you can set I18n.enforce_available_locales = false to avoid this message.
System information
System: Debian testing
Current User: git
Using RVM: no
Ruby Version: 2.1.2p95
Gem Version: 2.2.2
Bundler Version:1.7.3
Rake Version: 10.3.2
Sidekiq Version:2.17.0
GitLab information
Version: 7.3.1
Revision: 1660aa2
Directory: /home/git/gitlab
DB Adapter: postgresql
URL: http://localhost:8081
HTTP Clone URL: http://localhost:8081/some-project.git
SSH Clone URL: git@localhost:some-project.git
Using LDAP: no
Using Omniauth: no
GitLab Shell
Version: 2.0.1
Repositories: /home/git/repositories/
Hooks: /home/git/gitlab-shell/hooks/
Git: /usr/bin/git
+ git log -n1 --pretty=oneline
1660aa23e3f6bea8e0de54a420e29953f6bd194f Version 7.3.1
+ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
[deprecated] I18n.enforce_available_locales will default to true in the future. If you really want to skip validation of your locale you can set I18n.enforce_available_locales = false to avoid this message.
Checking Environment ...
Git configured for git user? ... yes
Checking Environment ... Finished
Checking GitLab Shell ...
GitLab Shell version >= 2.0.0 ? ... OK (2.0.1)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
Satellites access is drwxr-x---? ... yes
hooks directories in repos are links: ...
1/1 ... ok
1/2 ... ok
1/3 ... repository is empty
1/4 ... repository is empty
1/5 ... ok
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: FAILED. code: 502
gitlab-shell self-check failed
Try fixing it:
Make sure GitLab is running;
Check the gitlab-shell configuration file:
sudo -u git -H editor /home/git/gitlab-shell/config.yml
Please fix the error above and rerun the checks.
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... no
Try fixing it:
sudo -u git -H RAILS_ENV=production bin/background_jobs start
For more information see:
doc/install/installation.md in section "Install Init Script"
see log/sidekiq.log for possible errors
Please fix the error above and rerun the checks.
Checking Sidekiq ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
Database contains orphaned UsersGroups? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ...
1/1 ... yes
1/2 ... yes
1/3 ... yes
1/4 ... yes
1/5 ... yes
Projects have satellites? ...
1/1 ... yes
1/2 ... yes
1/3 ... can't create, repository is empty
1/4 ... can't create, repository is empty
1/5 ... yes
Redis version >= 2.0.0? ... yes
Ruby version >= 2.0.0 ? ... yes (2.1.2)
Your git bin path is "/usr/bin/git"
Git version >= 1.7.10 ? ... yes (2.1.0)
Checking GitLab ... Finished
My local Debian Testing server works fine. The system in question is using a near-identical setup, but originally used Omnibus on Debian Wheezy, before I switched the system to Jessie and was forced to make the move back to the "classic" gitlab. The only major deviations from the installation script is that I use lighttpd as the web server, and restored the Omnibus backup rather than installing a fresh database.
I would love to be able to help debug this, but Ruby is perhaps the only language still foreign to me.