Upgrading from 8.13.3 to 8.14.4 with invalid utf-8 in commit messages causes the migration to fail
I use the omnibus package, one of my repositories is a potentially bastardised linux kernel, one of the names has invalid utf-8 in a commit message.
Recipe: gitlab::database_migrations
* bash[migrate gitlab-rails database] action run
[execute] rake aborted!
StandardError: An error has occurred, all later migrations canceled:
source sequence is illegal/malformed utf-8/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20161124141322_migrate_process_commit_worker_jobs.rb:62:in `block in up'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/sidekiq-4.2.1/lib/sidekiq.rb:94:in `block in redis'
So it's failing at JSON.dumps(payload)
, I know nothing about ruby, so forgive the copy/paste solution from another issue, but I tried cleaning it up:
hash = {
id: commit.oid,
message: commit.message.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "?"),
parent_ids: commit.parent_ids,
authored_date: commit.author[:time],
author_name: commit.author[:name].encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "?"),
author_email: commit.author[:email].encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "?"),
committed_date: commit.committer[:time],
committer_email: commit.committer[:email].encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "?"),
committer_name: commit.committer[:name].encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "?")
}
That takes a very long time to run, I suppose the method of cleaning strings is poor. After that I got a new error:
wrong argument type Hash (expected String)/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/rugged-0.24.0/lib/rugged/repository.rb:99:in `lookup'
Edit: Running reconfigure again after stopping/starting gitlab got past the 'Hash' error.
I'm not sure how accurate this is, my installation was/is in a partially upgraded state.
dev@dev:~$ sudo gitlab-rake gitlab:env:info RAILS_ENV=production
System information
System: Ubuntu 16.04
Current User: git
Using RVM: no
Ruby Version: 2.3.1p112
Gem Version: 2.6.6
Bundler Version:1.13.6
Rake Version: 10.5.0
Sidekiq Version:4.2.1
GitLab information
Version: 8.14.4
Revision: 3ea6c8d
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
URL: http://dev.touchstar.co.uk
HTTP Clone URL: http://dev.touchstar.co.uk/some-group/some-project.git
SSH Clone URL: git@dev.touchstar.co.uk:some-group/some-project.git
Using LDAP: no
Using Omniauth: yes
Omniauth Providers:
GitLab Shell
Version: 4.0.3
Repository storage paths:
- default: /home/git/git-data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks/
Git: /opt/gitlab/embedded/bin/git