Unable to upgrade 9.5.2-ce to 9.5.2-ee relation "geo_nodes" does not exist
Summary
Unable to upgrade 9.5.2-ce to 9.5.2-ee
Steps to reproduce
Update package sources to point to ee repo
sudo apt install gitlab-ee
What is the current bug behavior?
Error 502 Whoops, GitLab is taking too much time to respond.
Relevant logs and/or screenshots
Recipe: gitlab::database_migrations
* bash[migrate gitlab-rails database] action run
[execute] WARNING: Unable to check/update clone_url_prefix for Geo: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist
LINE 5: WHERE a.attrelid = '"geo_nodes"'::regclass
^
: SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"geo_nodes"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 1
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
================================================================================
Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20170829-13466-197dc5f" ----
STDOUT: WARNING: Unable to check/update clone_url_prefix for Geo: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist
LINE 5: WHERE a.attrelid = '"geo_nodes"'::regclass
^
: SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"geo_nodes"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 1
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" "/tmp/chef-script20170829-13466-197dc5f" ----
Ran "bash" "/tmp/chef-script20170829-13466-197dc5f" returned 1
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb
51: bash "migrate gitlab-rails database" do
52: code <<-EOH
53: set -e
54: log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
55: umask 077
56: /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
57: STATUS=${PIPESTATUS[0]}
58: echo $STATUS > #{db_migrate_status_file}
59: exit $STATUS
60: EOH
61: environment env_variables unless env_variables.empty?
62: notifies :run, 'execute[enable pg_trgm extension]', :before if omnibus_helper.service_enabled?('postgresql')
63: notifies :run, "execute[clear the gitlab-rails cache]", :immediately
64: dependent_services.each do |svc|
65: notifies :restart, svc, :immediately
66: end
67: not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
68: only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
69: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:51:in `from_file'
bash("migrate gitlab-rails database") do
action [:run]
updated true
retries 0
retry_delay 2
default_guard_interpreter :default
command "migrate gitlab-rails database"
backup 5
returns 0
code " set -e\n log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n umask 077\n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n STATUS=${PIPESTATUS[0]}\n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-65f8414\n exit $STATUS\n"
interpreter "bash"
declared_type :bash
cookbook_name "gitlab"
recipe_name "database_migrations"
not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-65f8414) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-65f8414 | grep -Fx 0)"
only_if { #code block }
end
Platform:
---------
x86_64-linux
Running handlers:
Running handlers complete
Chef Client failed. 9 resources updated in 15 seconds
Ensuring PostgreSQL is updated:/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Checking for an omnibus managed postgresql: OK
Checking if we already upgraded: OK
The latest version 9.6.3 is already running, nothing to do
Ensuring PostgreSQL is updated: OK
Restarting previously running GitLab services
ok: run: gitaly: (pid 13980) 1s
ok: run: gitlab-monitor: (pid 13989) 0s
ok: run: gitlab-workhorse: (pid 13992) 0s
ok: run: logrotate: (pid 14001) 0s
ok: run: nginx: (pid 14007) 0s
ok: run: node-exporter: (pid 14050) 1s
ok: run: postgres-exporter: (pid 14056) 0s
ok: run: postgresql: (pid 13893) 13s
ok: run: redis: (pid 13453) 21s
ok: run: redis-exporter: (pid 14065) 1s
ok: run: sidekiq: (pid 14071) 0s
ok: run: unicorn: (pid 14078) 0s
Upgrade complete! If your GitLab server is misbehaving try running
sudo gitlab-ctl restart
before anything else. If you need to roll back to the previous version you can
use the database backup made during the upgrade (scroll up for the filename).´´´
#### Results of GitLab environment info
```WARNING: Unable to check/update clone_url_prefix for Geo: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist
LINE 5: WHERE a.attrelid = '"geo_nodes"'::regclass
^
: SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"geo_nodes"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 1
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
Tasks: TOP => gitlab:env:info => environment
(See full trace by running task with --trace)```
#### Results of GitLab application Check
```WARNING: Unable to check/update clone_url_prefix for Geo: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist
LINE 5: WHERE a.attrelid = '"geo_nodes"'::regclass
^
: SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"geo_nodes"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT 1
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
PG::UndefinedTable: ERROR: relation "licenses" does not exist
LINE 1: SELECT "licenses".* FROM "licenses" ORDER BY "licenses"."i...
^
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:207:in `load_license'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:189:in `current'
/opt/gitlab/embedded/service/gitlab-rails/app/models/license.rb:193:in `feature_available?'
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/gitlab/ldap/config.rb:9:in `enabled_extras?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/config.rb:30:in `available_servers'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:3:in `<module:Strategies>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/omniauth.rb:2:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:5:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in `load'
/opt/gitlab/embedded/bin/bundle:22:in `<main>'
Tasks: TOP => gitlab:check => gitlab:gitlab_shell:check => environment
(See full trace by running task with --trace)Edited by Stan Hu