Error 500 after migration to 8.6.1 error in 20160301124843 AddVisibilityLevelToGroups
Hi, After migration and server reboot we are seeing the following error (see below). I have checked gitlab-rake gitlab:check
and I only see database migrations failed.
This is an Omnibus install on CentOS 7.1 with embedded database.
All migrations up? ...
no
Try fixing it:
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
Please fix the error above and rerun the checks.
Database contains orphaned GroupMembers? ... no
When I try to run: gitlab-rake db:migrate RAILS_ENV=production --trace
I get the following error:
root@gitlab ~]# gitlab-rake db:migrate RAILS_ENV=production --trace
** Invoke db:migrate (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:migrate
== 20160301124843 AddVisibilityLevelToGroups: migrating =======================
-- add_column(:namespaces, :visibility_level, :integer, {:null=>false, :default=>20})
-> 0.1287s
-- add_index(:namespaces, :visibility_level)
-> 0.0084s
-- select_one("SELECT restricted_visibility_levels FROM application_settings ORDER BY id DESC LIMIT 1")
-> 0.0003s
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:
undefined method `<' for nil:NilClass/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20160301124843_add_visibility_level_to_groups.rb:8:in `up'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:611:in `exec_migration'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
/opt/gitlab/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:591:in `block in migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:590:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:768:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:1044:in `block in ddl_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/transactions.rb:220:in `transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:1044:in `ddl_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:959:in `block in migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:955:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:955:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:823:in `up'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:801:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/tasks/database_tasks.rb:137:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/rake:23:in `load'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/rake:23:in `<main>'
NoMethodError: undefined method `<' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20160301124843_add_visibility_level_to_groups.rb:8:in `up'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:611:in `exec_migration'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
/opt/gitlab/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:591:in `block in migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:590:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:768:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:1044:in `block in ddl_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/transactions.rb:220:in `transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:1044:in `ddl_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:959:in `block in migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:955:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:955:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:823:in `up'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/migration.rb:801:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/tasks/database_tasks.rb:137:in `migrate'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/activerecord-4.2.5.2/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/rake:23:in `load'
/opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/rake:23:in `<main>'
Tasks: TOP => db:migrate
This has broken the system completely. Both Mattermost and Gitlab are not properly working. Please help.