Bitbucket Server importer fails without error on proxy misconfiguration
Summary
When trying to import a repository from Bitbucket Server to Gitlab SaaS, the importer fails without error if there is a proxy misconfiguration for the Bitbucket Server. The project is created, but none of the data is cloned or copied.
It seems the issue is two-fold:
- The GitLab importer is configured to not follow http redirects when cloning. Running
git clone
on my local device works fine unless I turn off the githttp.followRedirects
configuration setting. This may be intentional behavior for Gitaly though for some reason I don't know. - The BitBucket Server proxy is misconfigured such that the link to clone returned by the BitBucket Server API has an
http:
protocol, despite cloning overhttp:
not being available (in our case will redirect tohttps:
). I imagine the URL used to clone the repository is being grabbed from the Bitbucket API.
The combination of these issues is what causes the error. The bigger problem, however, is that no error or remediation steps are communicated to the user. When using Gitlab SaaS, the user has no access to the server logs to see why the import failed.
Steps to reproduce
- Set up a hosted Bitbucket Server instance.
- Route traffic to the instance using a reverse proxy.
- Don't configure the Bitbucket Tomcat server according to these instructions.
- Create a repository in BitBucket.
- Import project in GitLab SaaS.
- Choose BitBucket Server.
- Input host and credentials for new server.
- Try to import the repository from BitBucket to GitLab SaaS.
- Get a failed import with no error.
What is the current bug behavior?
The import fails with no error or remediation messages.
What is the expected correct behavior?
One or both of the following behaviors would solve the issue:
- Allow the import clone to follow http redirects.
- Show the user an error indicating that cloning the repository failed with a 301 HTTP code.
Relevant logs and/or screenshots
Error in log (from reproducing the error on a self-managed instance), domain redacted:
{
"severity":"ERROR",
"time":"2021-04-30T16:00:54.774Z",
"correlation_id":"01F4HR5V5F1E6JA2XVAMRHTSZF",
"exception.class":"Gitlab::Git::CommandError",
"exception.message":"2:fetch remote: \"fatal: unable to access 'http://<redacted.domain>/scm/leg/adm.git/': The requested URL returned error: 301\\n\": exit status 128.",
"exception.backtrace":[
"lib/gitlab/git/wraps_gitaly_errors.rb:15:in `rescue in wrapped_gitaly_errors'",
"lib/gitlab/git/wraps_gitaly_errors.rb:6:in `wrapped_gitaly_errors'",
"lib/gitlab/git/repository.rb:806:in `fetch_remote'",
"lib/gitlab/metrics/instrumentation.rb:162:in `method_missing'",
"app/models/repository.rb:947:in `fetch_as_mirror'",
"lib/gitlab/metrics/instrumentation.rb:162:in `fetch_as_mirror'",
"lib/gitlab/bitbucket_server_import/importer.rb:145:in `import_repository'",
"lib/gitlab/bitbucket_server_import/importer.rb:51:in `execute'",
"app/services/projects/import_service.rb:125:in `import_data'",
"lib/gitlab/metrics/instrumentation.rb:162:in `import_data'",
"app/services/projects/import_service.rb:23:in `execute'",
"app/services/concerns/measurable.rb:35:in `execute'",
"app/workers/repository_import_worker.rb:26:in `perform'",
"ee/app/workers/ee/repository_import_worker.rb:9:in `perform'",
"lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'",
"lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:40:in `perform'",
"lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'",
"lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'",
"lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `block in call'",
"lib/gitlab/application_context.rb:70:in `block in use'",
"lib/gitlab/application_context.rb:70:in `use'",
"lib/gitlab/application_context.rb:27:in `with_context'",
"lib/gitlab/sidekiq_middleware/worker_context/server.rb:15:in `call'",
"lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'",
"lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'",
"lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'",
"lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:27:in `call'",
"lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'",
"lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'",
"lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'",
"lib/gitlab/with_request_store.rb:17:in `enabling_request_store'",
"lib/gitlab/with_request_store.rb:10:in `with_request_store'",
"lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:25:in `block in call'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:48:in `block in instrument'",
"lib/gitlab/metrics/background_transaction.rb:30:in `run'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:48:in `instrument'",
"ee/lib/ee/gitlab/sidekiq_middleware/server_metrics.rb:17:in `instrument'",
"lib/gitlab/sidekiq_middleware/server_metrics.rb:24:in `call'",
"lib/gitlab/sidekiq_middleware/monitor.rb:8:in `block in call'",
"lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'",
"lib/gitlab/sidekiq_middleware/monitor.rb:7:in `call'",
"lib/gitlab/sidekiq_logging/structured_logger.rb:19:in `call'"
],
"user.username":"root",
"tags.program":"sidekiq",
"tags.locale":"en",
"tags.feature_category":"importers",
"tags.correlation_id":"01F4HR5V5F1E6JA2XVAMRHTSZF",
"extra.sidekiq":{
"class":"RepositoryImportWorker",
"args":[
"35"
],
"retry":false,
"queue":"repository_import",
"version":0,
"backtrace":5,
"dead":false,
"status_expiration":54000,
"memory_killer_memory_growth_kb":50,
"memory_killer_max_memory_growth_kb":300000,
"jid":"148b144c6df4e8d2dffa7a69",
"created_at":1619798454.4736233,
"meta.user":"root",
"meta.project":"root/ADM",
"meta.root_namespace":"root",
"meta.caller_id":"Import::BitbucketServerController#create",
"meta.remote_ip":"172.17.0.1",
"meta.related_class":"Projects::CreateService",
"meta.feature_category":"importers",
"meta.client_id":"user/1",
"correlation_id":"01F4HR5V5F1E6JA2XVAMRHTSZF",
"enqueued_at":1619798454.4745195
},
"extra.project_path":"root/ADM",
"extra.importer":"bitbucket_server"
}
The first two attempts here illustrate the issue. The third attempt was to illustrate that an error mechanism is already in place for other expected errors.
Output of checks
This bug happens on GitLab.com