Setting up two factor authentication (2FA) always yields invalid pin for some users

Summary

Some (not all) users always get "Invalid Pin" when setting up 2FA.

Steps to reproduce

2FA is enforced

• Log in
• scan the QR code (tried with FreeOTP, Authy, Microsoft Authenticator on two iPhones)
• enter the given PIN
• Invalid PIN feedback is given

workaround, works for most people

• Log in
• enter the code manually
• enter the given PIN
• 2FA is enabled

Example Project

Not available.

What is the current bug behavior?

The PIN is not accepted for enabling 2FA.

What is the expected correct behavior?

The PIN is accepted for enabling 2FA.

Relevant logs and/or screenshots

production.log

Started POST "/profile/two_factor_auth" for xxx.yyy.xxx.yyy at 2020-09-10 07:58:16 +0200
Processing by Profiles::TwoFactorAuthsController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "pin_code"=>"011423"}
Completed 200 OK in 322ms (Views: 41.2ms | ActiveRecord: 6.3ms | Elasticsearch: 0.0ms | Allocations: 111318)
...
Redirected to https://gitlab.we.nl/profile/two_factor_auth
Filter chain halted as :check_two_factor_requirement rendered or redirected
Completed 302 Found in 14ms (ActiveRecord: 2.8ms | Elasticsearch: 0.0ms | Allocations: 2418)
Redirected to https://gitlab.we.nl/profile/two_factor_auth
Filter chain halted as :check_two_factor_requirement rendered or redirected

Output of checks

This bug happens on premise.

Results of GitLab environment info

Expand for output related to GitLab environment info

Url's are left out.

System information
System:		Ubuntu 18.04
Current User:	git
Using RVM:	no
Ruby Version:	2.6.6p146
Gem Version:	2.7.10
Bundler Version:1.17.3
Rake Version:	12.3.3
Redis Version:	5.0.9
Git Version:	2.28.0
Sidekiq Version:5.2.9
Go Version:	unknown

GitLab information
Version:	13.3.5
Revision:	467cb4161ad
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	11.7
URL:		https://
HTTP Clone URL:	https://xxx/some-group/some-project.git
SSH Clone URL:	git@xx:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	yes
Omniauth Providers: 

GitLab Shell
Version:	13.6.0
Repository storage paths:
- default: 	somefolder/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.6.0 ? ... OK (13.6.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 18 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
6/6 ... yes
15/7 ... yes
15/8 ... yes
15/9 ... yes
15/10 ... yes
19/11 ... yes
21/12 ... yes
24/13 ... yes
14/14 ... yes
30/15 ... yes
33/17 ... yes
34/18 ... yes
36/19 ... yes
2/20 ... yes
38/21 ... yes
39/22 ... yes
45/23 ... yes
45/24 ... yes
39/25 ... yes
5/27 ... yes
38/28 ... yes
14/30 ... yes
43/31 ... yes
46/32 ... yes
46/33 ... yes
15/34 ... yes
46/35 ... yes
47/44 ... yes
49/45 ... yes
52/46 ... yes
46/47 ... yes
53/48 ... yes
55/49 ... yes
46/50 ... yes
49/51 ... yes
49/52 ... yes
47/53 ... yes
58/54 ... yes
15/55 ... yes
15/56 ... yes
66/57 ... yes
67/58 ... yes
68/59 ... yes
69/60 ... yes
70/61 ... yes
71/63 ... yes
28/64 ... yes
46/65 ... yes
72/66 ... yes
73/67 ... yes
73/68 ... yes
73/69 ... yes
73/70 ... yes
73/71 ... yes
73/72 ... yes
73/73 ... yes
73/74 ... yes
73/75 ... yes
15/76 ... yes
46/77 ... yes
59/78 ... yes
74/79 ... yes
46/82 ... yes
74/83 ... yes
79/84 ... yes
14/85 ... yes
47/86 ... yes
82/87 ... yes
46/88 ... yes
79/89 ... yes
59/90 ... yes
46/91 ... yes
79/92 ... yes
15/93 ... yes
84/94 ... yes
15/95 ... yes
86/96 ... yes
86/97 ... yes
87/98 ... yes
3/99 ... yes
91/100 ... yes
91/101 ... yes
91/102 ... yes
91/103 ... yes
91/104 ... yes
79/105 ... yes
130/106 ... yes
93/107 ... yes
46/108 ... yes
94/109 ... yes
97/110 ... yes
97/111 ... yes
46/112 ... yes
46/113 ... yes
99/114 ... yes
100/115 ... yes
98/116 ... yes
101/117 ... yes
14/119 ... yes
98/120 ... yes
105/121 ... yes
106/122 ... yes
98/125 ... yes
103/127 ... yes
103/128 ... yes
104/130 ... yes
104/131 ... yes
105/132 ... yes
105/133 ... yes
22/134 ... yes
106/135 ... yes
106/136 ... yes
107/137 ... yes
5/138 ... yes
46/139 ... yes
3/140 ... yes
103/141 ... yes
105/142 ... yes
104/143 ... yes
106/144 ... yes
98/145 ... yes
15/146 ... yes
111/147 ... yes
17/148 ... yes
46/149 ... yes
102/150 ... yes
74/151 ... yes
22/152 ... yes
112/153 ... yes
114/154 ... yes
114/155 ... yes
116/156 ... yes
113/158 ... yes
46/159 ... yes
112/160 ... yes
117/161 ... yes
117/162 ... yes
17/163 ... yes
46/164 ... yes
120/165 ... yes
121/166 ... yes
121/167 ... yes
121/168 ... yes
121/169 ... yes
121/170 ... yes
121/171 ... yes
121/172 ... yes
121/173 ... yes
121/174 ... yes
121/175 ... yes
121/176 ... yes
121/177 ... yes
121/178 ... yes
121/179 ... yes
121/180 ... yes
121/181 ... yes
52/183 ... yes
15/185 ... yes
59/186 ... yes
59/187 ... yes
17/188 ... yes
103/189 ... yes
123/190 ... yes
124/191 ... yes
124/192 ... yes
71/193 ... yes
3/194 ... yes
3/195 ... yes
74/196 ... yes
71/197 ... yes
49/198 ... yes
46/199 ... yes
3/200 ... yes
126/201 ... yes
102/202 ... yes
127/204 ... yes
127/205 ... yes
46/206 ... yes
15/207 ... yes
129/209 ... yes
121/210 ... yes
121/211 ... yes
131/212 ... yes
46/213 ... yes
49/214 ... yes
132/215 ... yes
116/216 ... yes
15/217 ... yes
132/218 ... yes
39/219 ... yes
132/220 ... yes
132/221 ... yes
116/222 ... yes
46/223 ... yes
129/224 ... yes
139/225 ... yes
3/226 ... yes
46/227 ... yes
15/228 ... yes
46/229 ... yes
15/230 ... yes
140/231 ... yes
126/232 ... yes
15/233 ... yes
146/235 ... yes
46/236 ... yes
145/237 ... yes
17/238 ... yes
148/240 ... yes
46/241 ... yes
5/242 ... yes
103/243 ... yes
122/244 ... yes
149/245 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.6)
Git version >= 2.24.0 ? ... yes (2.28.0)
Git user has default SSH configuration? ... yes
Active users: ... 15
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished

Possible fixes

No fix available yet.

Additional: what happend before

We were experiencing problems after upgrading to 13.3.4 due to issue #244638 (closed). Therefore followed these steps:

1. disable 2FA on 13.3.4 sudo gitlab-rake gitlab:two_factor:disable_for_all_users
2. downgrade to 13.3.2 as suggested in the issue
3. re-enable 2FA
4. upgrade to 13.3.5 once it was released

For my account (which works fine; also re-enabling 2FA), 2FA was enabled on 13.3.2. The account which always yields "Invalid PIN" was not re-enabled on 13.3.2 but on 13.3.5.