authorized_keys file is not being updated, users can't use new keys and new users broken

Summary

authorized_keys file is not being updated when users add new keys or when new users are added

Steps to reproduce

As an existing GitLab user, add a new SSH key to your account. Try to use this ssh key to pull or push code.

Note: It does not matter if the key is ed25519 or RSA and the key size has impact either.

Example Project

Private / Self-Hosted Omnibus Install

What is the current bug behaviour?

• Client side: Access denied to pull or push repos
• Sever side: authorized_keys file has not been updated

What is the expected correct behaviour?

User should be able to pull/push with new key.

Relevant logs and/or screenshots

gitlab-rake check
...

# (exits 0)

and no errors

sudo gitlab-rake gitlab:shell:setup
# (exits 0)

However the authorized_keys file is not updated, if you first delete it, it is not recreated.

Results of GitLab environment info

Expand for output related to GitLab environment info

System information
System:
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.3.3p222
Gem Version:	2.6.6
Bundler Version:1.13.7
Rake Version:	10.5.0
Redis Version:	3.2.5
Git Version:	2.13.0
Sidekiq Version:5.0.0
Go Version:	unknown
GitLab information
Version:	9.3.5-ee
Revision:	44d109d
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
DB Version:	9.6.3
URL:		https://gitlab.office.infoxchange.net.au
HTTP Clone URL:	https://gitlab.office.infoxchange.net.au/some-group/some-project.git
SSH Clone URL:	git@gitlab.office.infoxchange.net.au:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	yes
Using Omniauth:	no
GitLab Shell
Version:	5.1.1
Repository storage paths:

• default: /mnt/repositories/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

Checking GitLab Shell ...
GitLab Shell version >= 5.1.1 ? ... OK (5.1.1)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
3/2 ... ok
4/3 ... ok
3/4 ... ok
3/5 ... ok
3/6 ... ok
3/7 ... ok
2/8 ... ok
3/9 ... ok
4/12 ... ok
4/13 ... ok
21/14 ... ok
4/16 ... ok
4/17 ... ok
21/18 ... ok
4/20 ... ok
2/22 ... ok
4/24 ... ok
4/25 ... ok
4/27 ... ok
4/29 ... ok
4/30 ... ok
4/32 ... ok
4/34 ... ok
4/35 ... ok
4/36 ... ok
14/37 ... repository is empty
4/38 ... ok
4/40 ... ok
4/41 ... ok
4/42 ... ok
4/43 ... ok
4/44 ... ok
3/47 ... ok
4/48 ... ok
3/50 ... ok
3/51 ... ok
21/53 ... repository is empty
4/55 ... ok
4/56 ... ok
4/57 ... ok
36/59 ... repository is empty
35/60 ... ok
4/61 ... ok
4/65 ... ok
4/66 ... ok
4/68 ... ok
4/69 ... ok
3/71 ... ok
39/72 ... ok
35/74 ... ok
39/77 ... ok
4/78 ... ok
14/79 ... ok
4/81 ... ok
4/84 ... ok
4/85 ... ok
4/86 ... ok
46/90 ... ok
46/91 ... ok
4/92 ... ok
4/93 ... ok
3/96 ... ok
3/97 ... ok
14/98 ... ok
3/100 ... ok
2/101 ... ok
3/104 ... ok
3/105 ... ok
3/106 ... ok
3/108 ... ok
4/109 ... ok
4/111 ... ok
4/112 ... ok
4/113 ... ok
4/114 ... ok
3/115 ... ok
39/116 ... ok
39/117 ... ok
48/118 ... ok
4/119 ... ok
39/120 ... ok
39/121 ... ok
4/122 ... ok
4/124 ... ok
4/131 ... ok
48/133 ... ok
4/134 ... ok
4/135 ... ok
21/136 ... ok
4/139 ... ok
21/140 ... ok
4/144 ... ok
21/145 ... repository is empty
4/146 ... ok
2/147 ... ok
3/149 ... ok
4/150 ... ok
21/151 ... ok
35/152 ... ok
14/153 ... ok
21/155 ... ok
7/156 ... ok
21/158 ... ok
4/163 ... ok
82/167 ... ok
82/168 ... ok
82/169 ... ok
4/170 ... ok
4/171 ... ok
3/173 ... ok
4/178 ... ok
35/179 ... ok
35/181 ... ok
35/182 ... ok
35/183 ... ok
35/184 ... ok
35/185 ... ok
35/186 ... ok
35/187 ... ok
35/188 ... ok
35/189 ... ok
35/190 ... ok
35/191 ... ok
35/192 ... ok
35/193 ... ok
35/194 ... ok
35/195 ... ok
35/196 ... ok
35/197 ... ok
35/198 ... ok
35/199 ... ok
35/200 ... ok
5/201 ... ok
35/202 ... ok
35/203 ... ok
29/204 ... ok
35/205 ... ok
46/206 ... ok
35/207 ... ok
35/208 ... ok
3/210 ... ok
35/212 ... ok
35/213 ... ok
29/215 ... ok
29/216 ... ok
35/217 ... ok
35/218 ... ok
4/219 ... ok
35/220 ... ok
4/221 ... ok
35/222 ... ok
35/223 ... ok
4/224 ... ok
3/225 ... ok
35/226 ... ok
35/227 ... ok
35/228 ... ok
35/229 ... ok
93/230 ... ok
35/232 ... ok
35/233 ... ok
35/234 ... ok
35/235 ... ok
35/238 ... ok
35/241 ... ok
35/242 ... ok
35/243 ... ok
3/244 ... ok
35/245 ... ok
35/246 ... ok
14/247 ... ok
35/249 ... ok
64/250 ... ok
35/251 ... ok
4/253 ... ok
5/259 ... ok
5/260 ... ok
64/264 ... ok
35/274 ... ok
35/275 ... ok
4/281 ... ok
35/283 ... ok
35/285 ... ok
35/286 ... ok
35/288 ... ok
35/290 ... ok
3/291 ... ok
35/292 ... ok
35/293 ... ok
35/294 ... ok
3/295 ... ok
3/300 ... ok
3/301 ... ok
4/302 ... ok
35/303 ... ok
35/304 ... ok
35/305 ... ok
35/306 ... ok
35/307 ... ok
103/308 ... ok
3/309 ... ok
35/310 ... ok
35/311 ... ok
5/312 ... ok
3/313 ... ok
3/314 ... ok
14/317 ... ok
5/318 ... ok
3/319 ... ok
35/321 ... ok
35/322 ... ok
103/323 ... ok
4/324 ... ok
7/325 ... ok
4/326 ... ok
35/327 ... ok
2/328 ... repository is empty
103/329 ... ok
3/331 ... repository is empty
103/332 ... repository is empty
35/339 ... ok
35/340 ... ok
35/342 ... ok
35/343 ... ok
35/351 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
IMAP server credentials are correct? ... yes
Init.d configured correctly? ... skipped (omnibus-gitlab has no init script)
MailRoom running? ... can't check because of previous errors
Checking Reply by email ... Finished
Checking LDAP ...
Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
< REDACTED >
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
3/2 ... yes
4/3 ... yes
3/4 ... yes
3/5 ... yes
3/6 ... yes
3/7 ... yes
2/8 ... yes
3/9 ... yes
4/12 ... yes
4/13 ... yes
21/14 ... yes
4/16 ... yes
4/17 ... yes
21/18 ... yes
4/20 ... yes
2/22 ... yes
4/24 ... yes
4/25 ... yes
4/27 ... yes
4/29 ... yes
4/30 ... yes
4/32 ... yes
4/34 ... yes
4/35 ... yes
4/36 ... yes
14/37 ... yes
4/38 ... yes
4/40 ... yes
4/41 ... yes
4/42 ... yes
4/43 ... yes
4/44 ... yes
3/47 ... yes
4/48 ... yes
3/50 ... yes
3/51 ... yes
21/53 ... yes
4/55 ... yes
4/56 ... yes
4/57 ... yes
36/59 ... yes
35/60 ... yes
4/61 ... yes
4/65 ... yes
4/66 ... yes
4/68 ... yes
4/69 ... yes
3/71 ... yes
39/72 ... yes
35/74 ... yes
39/77 ... yes
4/78 ... yes
14/79 ... yes
4/81 ... yes
4/84 ... yes
4/85 ... yes
4/86 ... yes
46/90 ... yes
46/91 ... yes
4/92 ... yes
4/93 ... yes
3/96 ... yes
3/97 ... yes
14/98 ... yes
3/100 ... yes
2/101 ... yes
3/104 ... yes
3/105 ... yes
3/106 ... yes
3/108 ... yes
4/109 ... yes
4/111 ... yes
4/112 ... yes
4/113 ... yes
4/114 ... yes
3/115 ... yes
39/116 ... yes
39/117 ... yes
48/118 ... yes
4/119 ... yes
39/120 ... yes
39/121 ... yes
4/122 ... yes
4/124 ... yes
4/131 ... yes
48/133 ... yes
4/134 ... yes
4/135 ... yes
21/136 ... yes
4/139 ... yes
21/140 ... yes
4/144 ... yes
21/145 ... yes
4/146 ... yes
2/147 ... yes
3/149 ... yes
4/150 ... yes
21/151 ... yes
35/152 ... yes
14/153 ... yes
21/155 ... yes
7/156 ... yes
21/158 ... yes
4/163 ... yes
82/167 ... yes
82/168 ... yes
82/169 ... yes
4/170 ... yes
4/171 ... yes
3/173 ... yes
4/178 ... yes
35/179 ... yes
35/181 ... yes
35/182 ... yes
35/183 ... yes
35/184 ... yes
35/185 ... yes
35/186 ... yes
35/187 ... yes
35/188 ... yes
35/189 ... yes
35/190 ... yes
35/191 ... yes
35/192 ... yes
35/193 ... yes
35/194 ... yes
35/195 ... yes
35/196 ... yes
35/197 ... yes
35/198 ... yes
35/199 ... yes
35/200 ... yes
5/201 ... yes
35/202 ... yes
35/203 ... yes
29/204 ... yes
35/205 ... yes
46/206 ... yes
35/207 ... yes
35/208 ... yes
3/210 ... yes
35/212 ... yes
35/213 ... yes
29/215 ... yes
29/216 ... yes
35/217 ... yes
35/218 ... yes
4/219 ... yes
35/220 ... yes
4/221 ... yes
35/222 ... yes
35/223 ... yes
4/224 ... yes
3/225 ... yes
35/226 ... yes
35/227 ... yes
35/228 ... yes
35/229 ... yes
93/230 ... yes
35/232 ... yes
35/233 ... yes
35/234 ... yes
35/235 ... yes
35/238 ... yes
35/241 ... yes
35/242 ... yes
35/243 ... yes
3/244 ... yes
35/245 ... yes
35/246 ... yes
14/247 ... yes
35/249 ... yes
64/250 ... yes
35/251 ... yes
4/253 ... yes
5/259 ... yes
5/260 ... yes
64/264 ... yes
35/274 ... yes
35/275 ... yes
4/281 ... yes
35/283 ... yes
35/285 ... yes
35/286 ... yes
35/288 ... yes
35/290 ... yes
3/291 ... yes
35/292 ... yes
35/293 ... yes
35/294 ... yes
3/295 ... yes
3/300 ... yes
3/301 ... yes
4/302 ... yes
35/303 ... yes
35/304 ... yes
35/305 ... yes
35/306 ... yes
35/307 ... yes
103/308 ... yes
3/309 ... yes
35/310 ... yes
35/311 ... yes
5/312 ... yes
3/313 ... yes
3/314 ... yes
14/317 ... yes
5/318 ... yes
3/319 ... yes
35/321 ... yes
35/322 ... yes
103/323 ... yes
4/324 ... yes
7/325 ... yes
4/326 ... yes
35/327 ... yes
2/328 ... yes
103/329 ... yes
3/331 ... yes
103/332 ... yes
35/339 ... yes
35/340 ... yes
35/342 ... yes
35/343 ... yes
35/351 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.3 ? ... yes (2.3.3)
Git version >= 2.7.3 ? ... yes (2.13.0)
Active users: ... 60
Elasticsearch version 5.1 - 5.3? ... skipped (elasticsearch is disabled)
Checking GitLab ... Finished

Note: I have tried:

• Upgrading from GitLab 9.3.4 to 9.3.5
• Running gitlab-ctl reconfigure
• Checking file permissions for the gitlab user (git)
• Regenerating the authorized_keys file
• Restarting the host