SSH mirroring to a custom port is broken

Summary

Testing https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2423 , I added an SSH repository mirror: ssh://git@altssh.gitlab.com:443/nick.thomas/gitlab-shell.git. Switching to master, I saw this in the sidekiq terminal output:

22:49:32 rails-background-jobs.1 | 2017-07-27T21:49:32.394Z 21009 TID-gq5ek7gwo WARN: RepositoryUpdateMirrorWorker::UpdateError: ssh: Could not resolve hostname altssh.gitlab.com:443: Name or service not known
22:49:32 rails-background-jobs.1 | fatal: Could not read from remote repository.

Steps to reproduce

Add an ssh://host:port URL to a mirror

What is the current bug behavior?

Unknown host error

What is the expected correct behavior?

Successful repository mirroring

Possible fixes

Fortunately, this works in !2423 (closed) . However, it may still be broken in push mirrors and elsewhere. At the least, we need to ensure this is covered by tests to avoid regressions.