SSH push/pull mirroring with password authentication on port 22 stalls on update/sync

Summary

SSH push and pull mirroring with password authentication on port 22 stalls on update/sync.

Tested using GDK. I've described push mirroring below but the same problem exists for pull mirroring in EE.

Steps to reproduce

  1. Configure sshd to use password authentication and port 22 e.g.:
AllowUsers GDK_USERNAME
AuthenticationMethods password
ListenAddress 127.0.0.1
PasswordAuthentication yes
UsePAM yes

Match User GDK_USERNAME
  1. Configure the source project to mirror a target project via push using a password:

mirrorb1

  1. Click the Update Now button

What is the current bug behavior?

The update icon keeps spinning but the update never takes place.

GDK logs show:

18:46:24 sshd.1                   | Connection from 127.0.0.1 port 42770 on 127.0.0.1 port 22

But nothing after. Interestingly, that's the same entry shown if I try to connect via an ssh client when it waits for a password.

So maybe the password isn't being passed to sshd correctly?

What is the expected correct behavior?

The source project is successfully mirrored

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:
Proxy:		no
Current User:	mark
Using RVM:	no
Ruby Version:	2.5.3p105
Gem Version:	2.7.6
Bundler Version:1.17.1
Rake Version:	12.3.1
Redis Version:	4.0.11
Git Version:	2.18.0
Sidekiq Version:5.2.3
Go Version:	go1.11 darwin/amd64

GitLab information
Version:	11.5.0-pre
Revision:	1c1a43fea15
Directory:	/Users/mark/dev/gitlab/gitlab-ee
DB Adapter:	postgresql
DB Version:	9.6.10
URL:		http://127.0.0.1:3000
HTTP Clone URL:	http://127.0.0.1:3000/some-group/some-project.git
SSH Clone URL:	mark@127.0.0.1:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers:


GitLab Shell
Version:	8.4.2
Repository storage paths:
  • default: /Users/mark/dev/gitlab/gdk-ee/repositories Hooks: /Users/mark/dev/gitlab/gdk-ee/gitlab-shell/hooks Git: /usr/local/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab Shell ...

GitLab Shell version >= 8.4.1 ? ... OK (8.4.2)
Running /Users/mark/dev/gitlab/gdk-ee/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK


Access to /Users/mark/dev/gitlab/gdk-ee/.ssh/authorized_keys: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Reply by email is disabled in config/gitlab.yml
Checking LDAP ...


LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... no
Try fixing it:
sudo chmod 700 /Users/mark/dev/gitlab/gitlab-ee/public/uploads
For more information see:
doc/install/installation.md in section "GitLab"
Please fix the error above and rerun the checks.
Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
Init script exists? ... no
Try fixing it:
Install the init script
For more information see:
doc/install/installation.md in section "Install Init Script"
Please fix the error above and rerun the checks.
Init script up-to-date? ... can't check because of previous errors
Projects have namespace: ...
Gitlab Org / Gitlab Test ... yes
Gitlab Org / Gitlab Shell ... yes
Gnuwget / Wget2 ... yes
Commit451 / Lab Coat ... yes
Jashkenas / Underscore ... yes
Flightjs / Flight ... yes
Twitter / Typeahead.Js ... yes
H5bp / Html5 Boilerplate ... yes
Administrator / target ... yes
Administrator / source ... yes
Administrator / target2 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.5.3)
Git version >= 2.9.5 ? ... yes (2.18.0)
Git user has default SSH configuration? ... no
Try fixing it:
mkdir ~/gitlab-check-backup-1543157663
sudo mv /Users/mark/.ssh/id_rsa ~/gitlab-check-backup-1543157663
sudo mv /Users/mark/.ssh/id_rsa.pub ~/gitlab-check-backup-1543157663
For more information see:
doc/ssh/README.md in section "SSH on the GitLab server"
Please fix the error above and rerun the checks.
Active users: ... 28
Elasticsearch version 5.1 - 5.5? ... skipped (elasticsearch is disabled)


Checking GitLab ... Finished

/cc @DouweM @nick.thomas