Skip to content

Improve error reporting in project import

What does this MR do?

Helps address this issue: #121732 (closed)

Prior to this change, exceptions raised in project import were only logged to production.log which is unstructured, so these errors could not be found in Sentry or easily in Kibana. This makes debugging issues like #121732 (closed) really hard.

This MR adds tracking of all exceptions from project import to Sentry and to exceptions_json.log -> using the method introduced in #32906 (closed). There is another issue open to go through and remove all instances of Rails.logger.(error|info): #209057 (closed)

After making these changes I receive the following in my development environment:

exceptions_json.log

{"severity":"ERROR","time":"2020-03-20T11:55:06.144Z","correlation_id":"V1lTYa0CHV2","extra.server":{"os":{"name":"Darwin","version":"Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64","build":"19.3.0","kernel_version":"Darwin fennec.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64"},"runtime":{"name":"ruby","version":"ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]"}},"extra.sidekiq":{"retry":false,"args":[30],"class":"RepositoryImportWorker","queue":"repository_import","backtrace":5,"status_expiration":54000,"memory_killer_memory_growth_kb":50,"memory_killer_max_memory_growth_kb":300000,"jid":"c675520af04f4ff21184c136","created_at":1584705303.7070348,"meta.user":"root","meta.caller_id":"Import::GitlabProjectsController#create","correlation_id":"V1lTYa0CHV2","uber-trace-id":"********","enqueued_at":1584705303.716969},"exception.class":"RuntimeError","exception.message":"this is an error in the after import service","exception.backtrace":["app/services/projects/after_import_service.rb:23:in `execute'","app/models/project_import_state.rb:61:in `block (3 levels) in \u003cclass:ProjectImportState\u003e'","lib/after_commit_queue.rb:39:in `instance_eval'","lib/after_commit_queue.rb:39:in `_run_after_commit_queue'","app/models/project.rb:1825:in `after_import'","ee/app/models/ee/project.rb:603:in `after_import'","app/workers/repository_import_worker.rb:36:in `perform'","ee/app/workers/ee/repository_import_worker.rb:9:in `perform'","lib/gitlab/with_request_store.rb:7:in `with_request_store'","lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'"]}

Sentry

Screen_Shot_2020-03-20_at_1.08.48_PM

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • ~~Label as security and @ mention @gitlab-com/gl-security/appsec~~
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Closes #121732 (closed)

Edited by 🤖 GitLab Bot 🤖

Merge request reports