Chef JSON ParserError when trying to upgrade EE from 8.11.7 to 8.12.3 Omnibus
Summary
When upgrading our 8.11.7-ee Omnibus installation to 8.12.3 via apt-get upgrade we encounter the following reproducible error (Actual behaviour)
Expected behavior
A smooth upgrade
Actual behavior
`gitlab-ctl upgrade Shutting down all GitLab services except those needed for migrations ok: down: gitlab-workhorse: 461s, normally up ok: down: logrotate: 461s, normally up ok: down: postgresql: 0s, normally up ok: down: redis: 1s, normally up ok: down: sidekiq: 456s, normally up ok: down: unicorn: 456s, normally up ok: run: postgresql: (pid 8523) 0s ok: run: redis: (pid 8531) 0s run: postgresql: (pid 8523) 0s; run: log: (pid 4468) 5540932s run: redis: (pid 8531) 0s; run: log: (pid 4470) 5540932s Reconfiguring GitLab to apply migrations Starting Chef Client, version 12.12.15 resolving cookbooks for run list: ["gitlab"] Synchronizing Cookbooks:
- runit (0.14.2)
- package (0.0.0)
- gitlab (0.0.1) Installing Cookbook Gems: Compiling Cookbooks...
================================================================================ Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
Chef::Exceptions::JSON::ParseError
parse error: trailing garbage ba45f34639953c394263b" } } etc/gitlab/gitlab.rb (right here) ------^
Cookbook Trace:
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/helper.rb:228:in read_gitlab_secrets' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/gitlab.rb:96:in generate_secrets'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/gitlab.rb:191:in generate_config' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:30:in from_file'
Relevant File Content:
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/helper.rb:
221:
222: class SecretsHelper
223:
224: def self.read_gitlab_secrets
225: existing_secrets ||= Hash.new
226:
227: if File.exists?("/etc/gitlab/gitlab-secrets.json")
228>> existing_secrets = Chef::JSONCompat.from_json(File.read("/etc/gitlab/gitlab-secrets.json"))
229: end
230:
231: existing_secrets.each do |k, v|
232: if Gitlab[k]
233: v.each do |pk, p|
234: # Note: Specifiying a secret in gitlab.rb will take precendence over "gitlab-secrets.json"
235: Gitlab[k][pk] ||= p
236: end
237: else
Platform:
x86_64-linux
Running handlers: Running handlers complete Chef Client failed. 0 resources updated in 02 seconds`
Checks
Cannot run check since DB isnt up to date:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "project_features" does not exist
Results of GitLab environment info
System: Debian 7.11 Current User: git Using RVM: no Ruby Version: 2.3.1p112 Gem Version: 2.6.6 Bundler Version:1.13.1 Rake Version: 10.5.0 Sidekiq Version:4.1.4