Project export fails v11.8.2
Overview
A blocking export issue was revealed in https://gitlab.com/gitlab-org/gitlab-ce/issues/59039#note_157036847. In the future, we should detect MRs with bad diffs and skip them rather than failing the entire export.
Original description
Following !25239 (merged) and #57579 (closed) the project exports now fail in v11.8.2 as it throws invalid base64
side note: I'm also experiencing issues with gitlab not closing issues that match the default regex pattern which I have enabled in /etc/gitlab/gitlab.rb
. Even though it is not in any way related I thought I should mention it.
2019-03-15_16:10:32.88256 2019-03-15T16:10:32.882Z 1584 TID-oribifhq4 WARN: {"context":"Job raised exception","job":{"class":"ProjectExportWorker","args":[2,6,null,{}],"retry":3,"queue":"project_export","backtrace":5,"jid":"efb183d942b9cf15d45f83c2","created_at":1552666222.536409,"correlation_id":"b0EbtMGQE66","enqueued_at":1552666222.5370896},"jobstr":"{\"class\":\"ProjectExportWorker\",\"args\":[2,6,null,{}],\"retry\":3,\"queue\":\"project_export\",\"backtrace\":5,\"jid\":\"efb183d942b9cf15d45f83c2\",\"created_at\":1552666222.536409,\"correlation_id\":\"b0EbtMGQE66\",\"enqueued_at\":1552666222.5370896}"}
2019-03-15_16:10:32.88257 2019-03-15T16:10:32.882Z 1584 TID-oribifhq4 WARN: Gitlab::ImportExport::Error: invalid base64
2019-03-15_16:10:32.88261 2019-03-15T16:10:32.882Z 1584 TID-oribifhq4 WARN: /opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_export/export_service.rb:75:in `cleanup_and_notify_error!'
2019-03-15_16:10:32.88261 /opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_export/export_service.rb:28:in `save_all!'
2019-03-15_16:10:32.88262 /opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_export/export_service.rb:9:in `execute'
2019-03-15_16:10:32.88262 /opt/gitlab/embedded/service/gitlab-rails/app/workers/project_export_worker.rb:14:in `perform'
2019-03-15_16:10:32.88262 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:185:in `execute_job'
2019-03-15_16:10:32.88263 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:167:in `block (2 levels) in process'
2019-03-15_16:10:32.88264 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
2019-03-15_16:10:32.88264 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
2019-03-15_16:10:32.88264 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:55:in `run'
2019-03-15_16:10:32.88265 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
2019-03-15_16:10:32.88265 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88265 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
2019-03-15_16:10:32.88266 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88266 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
2019-03-15_16:10:32.88266 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/correlation_id.rb:15:in `use_id'
2019-03-15_16:10:32.88266 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
2019-03-15_16:10:32.88267 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88267 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
2019-03-15_16:10:32.88267 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88268 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
2019-03-15_16:10:32.88268 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88268 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/shutdown.rb:54:in `call'
2019-03-15_16:10:32.88268 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88269 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/integrations/sidekiq.rb:9:in `call'
2019-03-15_16:10:32.88269 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2019-03-15_16:10:32.88269 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/middleware/chain.rb:133:in `invoke'
2019-03-15_16:10:32.88269 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:166:in `block in process'
2019-03-15_16:10:32.88270 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
2019-03-15_16:10:32.88270 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/job_retry.rb:108:in `local'
2019-03-15_16:10:32.88271 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
2019-03-15_16:10:32.88271 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/rails.rb:43:in `block in call'
2019-03-15_16:10:32.88272 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/activesupport-5.0.7.1/lib/active_support/execution_wrapper.rb:85:in `wrap'
2019-03-15_16:10:32.88272 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/activesupport-5.0.7.1/lib/active_support/reloader.rb:68:in `block in wrap'
2019-03-15_16:10:32.88272 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/activesupport-5.0.7.1/lib/active_support/execution_wrapper.rb:85:in `wrap'
2019-03-15_16:10:32.88272 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/activesupport-5.0.7.1/lib/active_support/reloader.rb:67:in `wrap'
2019-03-15_16:10:32.88273 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/rails.rb:42:in `call'
2019-03-15_16:10:32.88273 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
2019-03-15_16:10:32.88274 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:243:in `stats'
2019-03-15_16:10:32.88274 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
2019-03-15_16:10:32.88274 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/job_logger.rb:8:in `call'
2019-03-15_16:10:32.88275 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
2019-03-15_16:10:32.88275 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/job_retry.rb:73:in `global'
2019-03-15_16:10:32.88275 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:125:in `block in dispatch'
2019-03-15_16:10:32.88275 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/logging.rb:48:in `with_context'
2019-03-15_16:10:32.88276 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
2019-03-15_16:10:32.88276 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:124:in `dispatch'
2019-03-15_16:10:32.88276 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:165:in `process'
2019-03-15_16:10:32.88277 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:83:in `process_one'
2019-03-15_16:10:32.88277 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/processor.rb:71:in `run'
2019-03-15_16:10:32.88277 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/util.rb:16:in `watchdog'
2019-03-15_16:10:32.88278 /opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sidekiq-5.2.5/lib/sidekiq/util.rb:25:in `block in safe_thread'
Edited by Jeremy Watson (ex-GitLab)