Migrations job error OpenSSL::Cipher::CipherError
Summary
migration job didn't complete on update 1.8.4 --> 1.9.0
Versions
- Chart: gitlab-1.9.0
- Platform:
- Self-hosted: Kubespray
- Kubernetes: (
kubectl version)- Client: 1.14
- Server: 1.12
- Helm: (
helm version)- Client: 2.11
- Server: 2.11
Relevant logs
# kubectl get jobs
NAME COMPLETIONS DURATION AGE
gitlab-migrations.9 0/1 21m 21m
# kubectl get pods -w
gitlab-migrations.9-sdb4c 1/1 Running 5 14m
gitlab-migrations.9-sdb4c 0/1 Error 5 15m
gitlab-migrations.9-sdb4c 0/1 CrashLoopBackOff 5 15m
gitlab-migrations.9-sdb4c 1/1 Running 6 18m
gitlab-migrations.9-sdb4c 1/1 Terminating 6 18m
gitlab-migrations.9-sdb4c 0/1 Terminating 6 18m
gitlab-migrations.9-sdb4c 0/1 Terminating 6 18m
gitlab-migrations.9-sdb4c 0/1 Terminating 6 18m
# kubectl logs gitlab-migrations.9-sdb4c -f
+ /scripts/set-config /var/opt/gitlab/templates /srv/gitlab/config
Begin parsing .erb files from /var/opt/gitlab/templates
Writing /srv/gitlab/config/gitlab.yml
Writing /srv/gitlab/config/database.yml
Writing /srv/gitlab/config/resque.yml
Copying other config files found in /var/opt/gitlab/templates
+ exec /scripts/wait-for-deps /scripts/db-migrate
Checking database connection and schema version
WARNING: This version of GitLab depends on gitlab-shell 9.1.0, but you're running Unknown. Please update gitlab-shell.
Database Schema
Current version: 20190506135400
Codebase version: 20190506135400
Checking database migrations are up-to-date
Checking for new migrations
WARNING: This version of GitLab depends on gitlab-shell 9.1.0, but you're running Unknown. Please update gitlab-shell.
Disable authorized keys write in the database
WARNING: This version of GitLab depends on gitlab-shell 9.1.0, but you're running Unknown. Please update gitlab-shell.
/srv/gitlab/vendor/bundle/ruby/2.5.0/gems/encryptor-3.0.0/lib/encryptor.rb:98:in `final': OpenSSL::Cipher::CipherError
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/encryptor-3.0.0/lib/encryptor.rb:98:in `crypt'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/encryptor-3.0.0/lib/encryptor.rb:49:in `decrypt'
from /srv/gitlab/lib/gitlab/crypto_helper.rb:27:in `aes256_gcm_decrypt'
from /srv/gitlab/app/models/concerns/token_authenticatable_strategies/encrypted.rb:45:in `get_token'
from /srv/gitlab/app/models/concerns/token_authenticatable_strategies/base.rb:27:in `ensure_token'
from /srv/gitlab/app/models/concerns/token_authenticatable_strategies/encrypted.rb:32:in `ensure_token'
from /srv/gitlab/app/models/concerns/token_authenticatable.rb:40:in `block in add_authentication_token_field'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:413:in `block in make_lambda'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:197:in `block (2 levels) in halting'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:601:in `block (2 levels) in default_terminator'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:600:in `catch'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:600:in `block in default_terminator'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:198:in `block in halting'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:507:in `block in invoke_before'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:507:in `each'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:507:in `invoke_before'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:130:in `run_callbacks'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activesupport-5.1.7/lib/active_support/callbacks.rb:827:in `_run_save_callbacks'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/callbacks.rb:340:in `create_or_update'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/persistence.rb:129:in `save'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/validations.rb:44:in `save'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/attribute_methods/dirty.rb:35:in `save'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/persistence.rb:271:in `update_attribute'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/railties-5.1.7/lib/rails/commands/runner/runner_command.rb:44:in `perform'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/railties-5.1.7/lib/rails/commands/runner/runner_command.rb:37:in `eval'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/railties-5.1.7/lib/rails/commands/runner/runner_command.rb:37:in `perform'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/railties-5.1.7/lib/rails/command/base.rb:63:in `perform'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/railties-5.1.7/lib/rails/command.rb:44:in `invoke'
from /srv/gitlab/vendor/bundle/ruby/2.5.0/gems/railties-5.1.7/lib/rails/commands.rb:16:in `<top (required)>'
from /srv/gitlab/bin/rails:4:in `require'
from /srv/gitlab/bin/rails:4:in `<main>'
Storing Migration Status
# helm list
NAME REVISION UPDATED STATUS CHART APP VERSION
gitlab 9 Sun May 26 12:17:11 2019 DEPLOYED gitlab-1.9.0 11.11.0