PG upgrade on 12.10 fails when using the analyze_new_cluster.sh file
Summary
While running gitlab-ctl pg-upgrade
on a 12.10 Geo primary I ran into a problem where running analyze_new_cluster.sh failed because user_home
was apparently nil
(but the file was on disk).
The reconfigure that runs before the pg-upgrade happens was successful, so I don't really see a reason for that to be nil
, maybe some sort of race condition?
Steps to reproduce
The problem is, I've reverted the upgrade, upgraded again and everything worked fine. Did that 10x more times and no error, so I didn't find a way to consistently reproduce this again.
What is the current bug behavior?
user_home = @attributes.dig(:gitlab, :postgresql, :home) || @attributes.dig(:postgresql, :home)
was nil for some reason
What is the expected correct behavior?
The PG upgrade should complete, or if the path was incorrect or the file wasn't generated the upgrade shouldn't fail leaving the deploy page still up (we only rescue for errors when running the script).
Relevant logs
Relevant logs
Database upgrade is complete, running analyze_new_cluster.sh Traceback (most recent call last): 9: from /opt/gitlab/embedded/bin/omnibus-ctl:23:in `' 8: from /opt/gitlab/embedded/bin/omnibus-ctl:23:in `load' 7: from /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/omnibus-ctl-0.6.0/bin/omnibus-ctl:31:in `' 6: from /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/omnibus-ctl-0.6.0/lib/omnibus-ctl.rb:746:in `run' 5: from /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/omnibus-ctl-0.6.0/lib/omnibus-ctl.rb:204:in `block in add_command_under_category' 4: from /opt/gitlab/embedded/service/omnibus-ctl/pg-upgrade.rb:219:in `block in load_file' 3: from /opt/gitlab/embedded/service/omnibus-ctl/pg-upgrade.rb:286:in `general_upgrade' 2: from /opt/gitlab/embedded/service/omnibus-ctl/pg-upgrade.rb:258:in `common_post_upgrade' 1: from /opt/gitlab/embedded/service/omnibus-ctl/pg-upgrade.rb:463:in `analyze_cluster' /opt/gitlab/embedded/service/omnibus-ctl/pg-upgrade.rb:463:in `realpath': no implicit conversion of nil into String (TypeError)
Details of package version
Provide the package version installation details
12.10.7
Environment details
- Operating System:
Ubuntu 18.04.3 LTS
- Installation Target, remove incorrect values:
- VM: GCP
- Installation Type, remove incorrect values:
- Upgrade from version
12.9.1
- Upgrade from version
- Is there any other software running on the machine:
no
- Is this a single or multiple node installation?
Geo setup, though this was the primary
- Resources
- CPU:
2 core
- Memory total:
8 GB
- CPU:
Configuration details
Provide the relevant sections of `/etc/gitlab/gitlab.rb`
# Standard Geo primary config external_url 'https://****/' nginx['redirect_http_to_https'] = false postgresql['sql_user_password'] = '****' gitlab_rails['db_password'] = '****' postgresql['listen_address'] = '0.0.0.0' postgresql['md5_auth_cidr_addresses'] = ['****/32', 'localhost'] roles ['geo_primary_role'] postgresql['max_replication_slots'] = 1 gitlab_rails['auto_migrate'] = true