Skip to content

GithubImporter: Fix "ArgumentError: string contains null byte"

What does this MR do?

Some github imports are failing with "ArgumentError: string contains null byte", to fix that we're now cleaning the comments imported with Gitlab::EncodingHelper#clean.

Related to: #330614 (closed)

example of exception
{
  "severity": "ERROR",
  "time": "2021-05-08T02:40:33.950Z",
  "correlation_id": "573874bb5174a590276799ee274da98a",
  "exception.class": "ArgumentError",
  "exception.message": "string contains null byte",
  "exception.backtrace": [
    "lib/gitlab/database.rb:178:in `block (2 levels) in bulk_insert'",
    "lib/gitlab/database.rb:177:in `map'",
    "lib/gitlab/database.rb:177:in `block in bulk_insert'",
    "lib/gitlab/database.rb:176:in `map'",
    "lib/gitlab/database.rb:176:in `bulk_insert'",
    "lib/gitlab/github_import/importer/note_importer.rb:41:in `execute'",
    "app/workers/concerns/gitlab/github_import/object_importer.rb:31:in `import'",
    "app/workers/concerns/gitlab/github_import/rescheduling_methods.rb:31:in `try_import'",
    "app/workers/concerns/gitlab/github_import/rescheduling_methods.rb:19:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:40:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'",
    "lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'",
    "lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `block in call'",
    "lib/gitlab/application_context.rb:63:in `block in use'",
    "lib/gitlab/application_context.rb:63:in `use'",
    "lib/gitlab/application_context.rb:24:in `with_context'",
    "lib/gitlab/sidekiq_middleware/worker_context/server.rb:15:in `call'",
    "lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'",
    "lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/admin_mode/server.rb:8:in `call'",
    "lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'",
    "lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'",
    "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'",
    "lib/gitlab/with_request_store.rb:17:in `enabling_request_store'",
    "lib/gitlab/with_request_store.rb:10:in `with_request_store'",
    "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:38:in `block in call'",
    "lib/gitlab/metrics/background_transaction.rb:30:in `run'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:38:in `call'",
    "lib/gitlab/sidekiq_middleware/monitor.rb:8:in `block in call'",
    "lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'",
    "lib/gitlab/sidekiq_middleware/monitor.rb:7:in `call'",
    "lib/gitlab/sidekiq_logging/structured_logger.rb:19:in `call'"
  ],

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Kassio Borges

Merge request reports