database upgrade script fails while upgrading to gitlab-ce-12.3.0 on centos
Per request by @yorickpeterse, who released this update.
I upgraded from the lastest 12.2.x to 12.3.0 on centos and ran into the problem below:
Most important error: NoMethodError: undefined method type' for nil:NilClass`
After a reboot and running gitlab-ctl reconfigure gitlab is running but the error remains.
Recipe: gitlab::database_migrations
* bash[migrate gitlab-rails database] action run
[execute] rake aborted!
StandardError: An error has occurred, all later migrations canceled:
undefined method `type' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1066:in `create_column_from'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:462:in `rename_column_concurrently'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190822175441_rename_epics_state_to_state_id.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
NoMethodError: undefined method `type' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1066:in `create_column_from'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:462:in `rename_column_concurrently'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190822175441_rename_epics_state_to_state_id.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20190822175441 RenameEpicsStateToStateId: migrating ========================
-- transaction_open?()
-> 0.0000s
-- columns(:epics)
-> 0.0020s
================================================================================
Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/root/.tmp/chef-script20190922-1044-mm0br0" ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:
undefined method `type' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1066:in `create_column_from'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:462:in `rename_column_concurrently'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190822175441_rename_epics_state_to_state_id.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
NoMethodError: undefined method `type' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1066:in `create_column_from'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:462:in `rename_column_concurrently'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190822175441_rename_epics_state_to_state_id.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20190822175441 RenameEpicsStateToStateId: migrating ========================
-- transaction_open?()
-> 0.0000s
-- columns(:epics)
-> 0.0020s
STDERR:
---- End output of "bash" "/root/.tmp/chef-script20190922-1044-mm0br0" ----
Ran "bash" "/root/.tmp/chef-script20190922-1044-mm0br0" returned 1
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb
54: bash "migrate gitlab-rails database" do
55: code <<-EOH
56: set -e
57: log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
58: umask 077
59: /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
60: STATUS=${PIPESTATUS[0]}
61: chown #{account_helper.gitlab_user}:#{account_helper.gitlab_group} ${log_file}
62: echo $STATUS > #{db_migrate_status_file}
63: exit $STATUS
64: EOH
65: environment env_variables unless env_variables.empty?
66: notifies :run, "execute[clear the gitlab-rails cache]", :immediately
67: dependent_services.each do |svc|
68: notifies :restart, svc, :immediately
69: end
70: not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
71: only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
72: end
73:
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:54:in `from_file'
bash("migrate gitlab-rails database") do
action [:run]
default_guard_interpreter :default
command nil
backup 5
interpreter "bash"
declared_type :bash
cookbook_name "gitlab"
recipe_name "database_migrations"
code " set -e\n log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n umask 077\n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n STATUS=${PIPESTATUS[0]}\n chown git:git ${log_file}\n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-7099ecf77cb\n exit $STATUS\n"
domain nil
user nil
not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-7099ecf77cb) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-7099ecf77cb | grep -Fx 0)"
only_if { #code block }
end
System Info:
------------
chef_version=14.13.11
platform=centos
platform_version=7.7.1908
ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
Running handlers:
There was an error running gitlab-ctl reconfigure:
bash[migrate gitlab-rails database] (gitlab::database_migrations line 54) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/root/.tmp/chef-script20190922-1044-mm0br0" ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:
undefined method `type' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1066:in `create_column_from'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:462:in `rename_column_concurrently'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190822175441_rename_epics_state_to_state_id.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
NoMethodError: undefined method `type' for nil:NilClass
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1066:in `create_column_from'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:462:in `rename_column_concurrently'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190822175441_rename_epics_state_to_state_id.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20190822175441 RenameEpicsStateToStateId: migrating ========================
-- transaction_open?()
-> 0.0000s
-- columns(:epics)
-> 0.0020s
STDERR:
---- End output of "bash" "/root/.tmp/chef-script20190922-1044-mm0br0" ----
Ran "bash" "/root/.tmp/chef-script20190922-1044-mm0br0" returned 1
Running handlers complete
Chef Client failed. 3 resources updated in 22 seconds