Skip to content

Runner decryption error, 500 on /admin/runners

Summary

Getting 500 errors on the /admin/runners page on a test instance today after a restore from backup.

The logs show:

OpenSSL::Cipher::CipherError:
        from /srv/gitlab/vendor/bundle/ruby/2.4.0/gems/encryptor-3.0.0/lib/encryptor.rb:98:in `final'
        from /srv/gitlab/vendor/bundle/ruby/2.4.0/gems/encryptor-3.0.0/lib/encryptor.rb:98:in `crypt'
        from /srv/gitlab/vendor/bundle/ruby/2.4.0/gems/encryptor-3.0.0/lib/encryptor.rb:49:in `decrypt'
        from /srv/gitlab/lib/gitlab/crypto_helper.rb:27:in `aes256_gcm_decrypt'
        from /srv/gitlab/app/models/concerns/token_authenticatable_strategies/encrypted.rb:55:in `get_token'
        from /srv/gitlab/app/models/concerns/token_authenticatable_strategies/base.rb:33:in `ensure_token!'
        from /srv/gitlab/app/models/concerns/token_authenticatable.rb:43:in `block in add_authentication_token_field'
        from /srv/gitlab/app/models/application_setting.rb:409:in `runners_registration_token'

I see the following on the rails console:

ApplicationSetting.current.read_attribute(:runners_registration_token_encrypted)
=> "E6lBZgGIhBHJDu5K+Vz21mQLj0QLY8KLBI1KK7/90CvvmKgI86XuuPQ1W4w8lAlAgdtdY2uqMh8WUt1xgK/x3YfPYLlhwk7PvgBauC4a4QI="

This is likely because the db_key_base does not match, as the old secrets were not backed up. But there doesn't appear to be an easy way to just reset the token from this state.