RepositoryUpdateMirrorWorker fails with undefined method `split' for 13:Integer

Summary

Pull Mirroring on a customer project was stuck updating for hours in the UI. The RepositoryUpdateMirrorWorker failed with the following error message:

"error_message": "undefined method `split' for 13:Integer",

"error_backtrace": [
        "lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'",
        "lib/gitlab/database/load_balancing/load_balancer.rb:103:in `block in read_write'",
        "lib/gitlab/database/load_balancing/load_balancer.rb:159:in `retry_with_backoff'",
        "lib/gitlab/database/load_balancing/load_balancer.rb:101:in `read_write'",
        "lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'",
        "lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction'",
        "lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'",
        "lib/gitlab/database/load_balancing/load_balancer.rb:103:in `block in read_write'",
        "lib/gitlab/database/load_balancing/load_balancer.rb:159:in `retry_with_backoff'",
        "lib/gitlab/database/load_balancing/load_balancer.rb:101:in `read_write'",
        "lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'",
        "lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction'",
        "lib/gitlab/database.rb:265:in `block in transaction'",
        "lib/gitlab/database.rb:264:in `transaction'",
        "app/models/project_import_state.rb:83:in `mark_as_failed'",
        "ee/app/workers/repository_update_mirror_worker.rb:57:in `fail_mirror'",
        "ee/app/workers/repository_update_mirror_worker.rb:36:in `rescue in perform'",
        "ee/app/workers/repository_update_mirror_worker.rb:19:in `perform'",
        "lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:24:in `call'",
        "lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:20:in `perform'",
        "lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:57:in `perform'",
        "lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'",
        "lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'",
        "lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `block in call'",
        "lib/gitlab/application_context.rb:74:in `block in use'",
        "lib/gitlab/application_context.rb:74:in `use'",
        "lib/gitlab/application_context.rb:27:in `with_context'",
        "lib/gitlab/sidekiq_middleware/worker_context/server.rb:15:in `call'",
        "lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'",
        "lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'",
        "lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'",
        "lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'",
        "lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'",
        "lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'",
        "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'",
        "lib/gitlab/with_request_store.rb:17:in `enabling_request_store'",
        "lib/gitlab/with_request_store.rb:10:in `with_request_store'",
        "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'",
        "lib/gitlab/sidekiq_middleware/server_metrics.rb:66:in `block in call'",
        "lib/gitlab/sidekiq_middleware/server_metrics.rb:89:in `block in instrument'",
        "lib/gitlab/metrics/background_transaction.rb:30:in `run'",
        "lib/gitlab/sidekiq_middleware/server_metrics.rb:89:in `instrument'",
        "lib/gitlab/sidekiq_middleware/server_metrics.rb:65:in `call'",
        "lib/gitlab/sidekiq_middleware/monitor.rb:8:in `block in call'",
        "lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'",
        "lib/gitlab/sidekiq_middleware/monitor.rb:7:in `call'",
        "lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'",
        "lib/gitlab/sidekiq_logging/structured_logger.rb:19:in `call'"

Retriggering the worker via the production console allowed the mirror to update. There also seems to be a number of failures clustered around the same time.

Steps to reproduce

Not able to reproduce

Example Project

• ZD Ticket: https://gitlab.zendesk.com/agent/tickets/238192

What is the current bug behavior?

RepositoryUpdateMirrorWorker fails with the error undefined method `split' for 13:Integer and causes the pull mirror to become stuck in a pending state.

What is the expected correct behavior?

RepositoryUpdateMirrorWorker should fail and allow you to force update the mirror again. The error is also unhelpful.

Relevant logs and/or screenshots

• https://log.gprd.gitlab.net/goto/304002a0623b8efb2132394e6ae82166
• https://log.gprd.gitlab.net/goto/075647cb81d649034f984e0c37ed7a4a

Output of checks

This happens on GitLab.com 14.4.0-pre c430f6b7

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)

Possible fixes