timeout and error 502 - seem to be related to one user

Summary

We have one user who gets timeouts when creating an issue or a comment. This looks like this:

image

However, when reloading the page, you see that the issue or the comment was instantly created.

Log:

2020-11-17T14:50:03.660853107Z source=rack-timeout id=Riv9FU0uu58 timeout=60000ms service=60000ms state=timed_out at=error

The GUI is still reloading and running into 502 after about 5min.


Also, when trying to set Todos triggered by that user to "Done", the GUI is running in timeout. This seems to be the case on different users - it just depends to the user from whom the todo was triggered.

Screenshot_20201117_144134

Log:

lib/gitlab/error_tracking.rb:52:in `with_context'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
source=rack-timeout id=bxBj8DDsWZ1 timeout=60000ms service=60000ms state=timed_out at=error

Results of GitLab environment info

Expand for output related to GitLab environment info
```
root@gitlab-bundle-5bc6dd6957-2mdnd:/# gitlab-rake gitlab:env:info

System information
System:
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   2.6.6p146
Gem Version:    2.7.10
Bundler Version:1.17.3
Rake Version:   12.3.3
Redis Version:  5.0.9
Git Version:    2.28.0
Sidekiq Version:5.2.9
Go Version:     unknown

GitLab information
Version:        13.4.0-ee
Revision:       e70802d39ca
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     11.9
URL:            https://git.example.org
HTTP Clone URL: https://git.example.org/some-group/some-project.git
SSH Clone URL:  ssh://git@git.example.org:32222/some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: 

GitLab Shell
Version:        13.7.0
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell
Git:            /opt/gitlab/embedded/bin/git
```

Results of GitLab application Check

Expand for output related to the GitLab application check
root@gitlab-bundle-5bc6dd6957-2mdnd:/# gitlab-rake gitlab:env:info

System information
System:
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   2.6.6p146
Gem Version:    2.7.10
Bundler Version:1.17.3
Rake Version:   12.3.3
Redis Version:  5.0.9
Git Version:    2.28.0
Sidekiq Version:5.2.9
Go Version:     unknown

GitLab information
Version:        13.4.0-ee
Revision:       e70802d39ca
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     11.9
URL:            https://git.reddoxx.com
HTTP Clone URL: https://git.reddoxx.com/some-group/some-project.git
SSH Clone URL:  ssh://git@git.reddoxx.com:32222/some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: 

GitLab Shell
Version:        13.7.0
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell
Git:            /opt/gitlab/embedded/bin/git
root@gitlab-bundle-5bc6dd6957-2mdnd:/# gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 13.7.0 ? ... OK (13.7.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ... 
4/2 ... yes
51/3 ... yes
5/7 ... yes
50/9 ... yes
4/10 ... yes
2/11 ... yes
4/12 ... yes
4/13 ... yes
4/14 ... yes
8/15 ... yes
8/16 ... yes
8/17 ... yes
8/18 ... yes
51/19 ... yes
51/20 ... yes
51/21 ... yes
51/22 ... yes
94/23 ... yes
8/24 ... yes
8/25 ... yes
8/26 ... yes
51/27 ... yes
8/28 ... yes
8/29 ... yes
8/30 ... yes
12/31 ... yes
10/32 ... yes
10/33 ... yes
10/34 ... yes
8/36 ... yes
51/37 ... yes
50/38 ... yes
11/39 ... yes
11/40 ... yes
11/41 ... yes
11/42 ... yes
11/43 ... yes
12/46 ... yes
12/48 ... yes
5/49 ... yes
13/50 ... yes
51/51 ... yes
14/52 ... yes
14/54 ... yes
14/55 ... yes
14/56 ... yes
14/57 ... yes
14/58 ... yes
4/59 ... yes
15/61 ... yes
12/62 ... yes
4/63 ... yes
5/64 ... yes
5/65 ... yes
12/66 ... yes
5/67 ... yes
17/68 ... yes
5/69 ... yes
8/70 ... yes
5/71 ... yes
5/72 ... yes
33/73 ... yes
5/74 ... yes
37/75 ... yes
8/76 ... yes
8/77 ... yes
37/78 ... yes
37/79 ... yes
19/80 ... yes
19/81 ... yes
19/82 ... yes
19/83 ... yes
19/84 ... yes
19/85 ... yes
19/86 ... yes
19/87 ... yes
20/88 ... yes
33/89 ... yes
37/92 ... yes
32/97 ... yes
25/99 ... yes
4/101 ... yes
25/105 ... yes
25/106 ... yes
25/107 ... yes
2/111 ... yes
2/113 ... yes
2/114 ... yes
32/115 ... yes
2/116 ... yes
25/118 ... yes
25/119 ... yes
17/124 ... yes
15/126 ... yes
33/132 ... yes
33/134 ... yes
4/135 ... yes
32/138 ... yes
33/139 ... yes
33/140 ... yes
33/141 ... yes
33/143 ... yes
35/144 ... yes
22/145 ... yes
35/146 ... yes
35/147 ... yes
35/148 ... yes
4/149 ... yes
5/150 ... yes
5/151 ... yes
22/153 ... yes
2/154 ... yes
14/155 ... yes
35/156 ... yes
22/158 ... yes
33/159 ... yes
5/162 ... yes
22/163 ... yes
36/164 ... yes
36/165 ... yes
36/166 ... yes
36/170 ... yes
36/171 ... yes
36/172 ... yes
36/173 ... yes
36/174 ... yes
36/175 ... yes
36/176 ... yes
36/177 ... yes
36/178 ... yes
36/179 ... yes
36/180 ... yes
36/181 ... yes
36/182 ... yes
36/183 ... yes
36/184 ... yes
36/185 ... yes
36/186 ... yes
4/187 ... yes
33/191 ... yes
32/193 ... yes
5/194 ... yes
22/198 ... yes
33/199 ... yes
14/200 ... yes
4/204 ... yes
33/205 ... yes
5/206 ... yes
33/207 ... yes
9/208 ... yes
4/209 ... yes
14/210 ... yes
4/211 ... yes
33/214 ... yes
36/215 ... yes
35/216 ... yes
35/218 ... yes
36/219 ... yes
40/222 ... yes
32/224 ... yes
33/231 ... yes
33/233 ... yes
42/234 ... yes
33/235 ... yes
32/237 ... yes
36/238 ... yes
36/239 ... yes
17/240 ... yes
36/241 ... yes
32/242 ... yes
40/243 ... yes
40/244 ... yes
51/246 ... yes
31/247 ... yes
14/249 ... yes
22/255 ... yes
33/257 ... yes
37/258 ... yes
33/259 ... yes
66/260 ... yes
36/261 ... yes
40/262 ... yes
33/265 ... yes
36/268 ... yes
43/269 ... yes
43/270 ... yes
43/271 ... yes
35/272 ... yes
31/273 ... yes
33/274 ... yes
35/278 ... yes
42/279 ... yes
42/280 ... yes
4/281 ... yes
43/282 ... yes
33/284 ... yes
33/285 ... yes
45/288 ... yes
4/289 ... yes
43/290 ... yes
43/291 ... yes
35/292 ... yes
43/293 ... yes
43/295 ... yes
33/296 ... yes
43/297 ... yes
43/298 ... yes
35/299 ... yes
22/300 ... yes
31/301 ... yes
43/302 ... yes
33/303 ... yes
43/304 ... yes
51/305 ... yes
43/306 ... yes
54/309 ... yes
36/310 ... yes
22/311 ... yes
43/313 ... yes
2/314 ... yes
45/316 ... yes
42/317 ... yes
43/318 ... yes
25/319 ... yes
43/320 ... yes
43/321 ... yes
43/322 ... yes
43/323 ... yes
45/324 ... yes
45/326 ... yes
43/327 ... yes
43/328 ... yes
14/329 ... yes
43/331 ... yes
43/333 ... yes
43/334 ... yes
5/335 ... yes
46/336 ... yes
43/337 ... yes
33/342 ... yes
43/343 ... yes
36/344 ... yes
2/345 ... yes
43/347 ... yes
14/348 ... yes
49/349 ... yes
45/350 ... yes
45/351 ... yes
46/352 ... yes
22/353 ... yes
43/354 ... yes
35/355 ... yes
43/356 ... yes
46/357 ... yes
46/359 ... yes
33/360 ... yes
4/361 ... yes
43/362 ... yes
9/363 ... yes
43/364 ... yes
43/365 ... yes
36/366 ... yes
45/367 ... yes
45/368 ... yes
43/369 ... yes
43/370 ... yes
45/372 ... yes
31/373 ... yes
33/374 ... yes
54/376 ... yes
33/377 ... yes
53/378 ... yes
33/379 ... yes
53/380 ... yes
53/381 ... yes
45/382 ... yes
15/384 ... yes
43/385 ... yes
45/386 ... yes
22/387 ... yes
32/388 ... yes
32/389 ... yes
43/390 ... yes
53/391 ... yes
22/392 ... yes
53/393 ... yes
53/394 ... yes
43/395 ... yes
53/396 ... yes
53/397 ... yes
22/398 ... yes
33/399 ... yes
14/401 ... yes
57/402 ... yes
57/403 ... yes
14/404 ... yes
2/405 ... yes
22/406 ... yes
57/408 ... yes
22/410 ... yes
51/411 ... yes
51/412 ... yes
53/413 ... yes
43/414 ... yes
43/415 ... yes
22/417 ... yes
22/418 ... yes
58/419 ... yes
45/420 ... yes
68/421 ... yes
58/423 ... yes
43/424 ... yes
43/425 ... yes
57/427 ... yes
58/428 ... yes
63/432 ... yes
63/433 ... yes
63/434 ... yes
63/436 ... yes
64/437 ... yes
63/438 ... yes
43/439 ... yes
64/440 ... yes
68/442 ... yes
32/443 ... yes
2/444 ... yes
43/445 ... yes
43/446 ... yes
43/447 ... yes
43/448 ... yes
25/450 ... yes
15/451 ... yes
33/452 ... yes
58/453 ... yes
43/454 ... yes
45/455 ... yes
43/456 ... yes
43/457 ... yes
43/458 ... yes
68/460 ... yes
51/461 ... yes
66/463 ... yes
43/464 ... yes
33/465 ... yes
33/466 ... yes
68/468 ... yes
43/469 ... yes
33/470 ... yes
68/471 ... yes
43/472 ... yes
43/473 ... yes
43/474 ... yes
33/475 ... yes
9/476 ... yes
43/477 ... yes
68/478 ... yes
68/479 ... yes
32/480 ... yes
22/481 ... yes
2/482 ... yes
43/483 ... yes
43/484 ... yes
66/485 ... yes
68/486 ... yes
43/487 ... yes
43/488 ... yes
35/489 ... yes
9/490 ... yes
43/491 ... yes
43/492 ... yes
43/493 ... yes
43/494 ... yes
33/495 ... yes
33/496 ... yes
43/497 ... yes
2/498 ... yes
4/499 ... yes
31/500 ... yes
22/501 ... yes
66/502 ... yes
49/503 ... yes
43/504 ... yes
35/505 ... yes
36/508 ... yes
36/510 ... yes
68/511 ... yes
68/512 ... yes
43/514 ... yes
14/517 ... yes
35/518 ... yes
43/519 ... yes
14/520 ... yes
2/521 ... yes
45/522 ... yes
72/523 ... yes
68/524 ... yes
72/525 ... yes
72/526 ... yes
72/527 ... yes
72/528 ... yes
72/529 ... yes
72/530 ... yes
72/531 ... yes
72/532 ... yes
45/533 ... yes
22/534 ... yes
80/535 ... yes
80/536 ... yes
80/537 ... yes
81/538 ... yes
81/539 ... yes
43/540 ... yes
22/541 ... yes
81/543 ... yes
81/544 ... yes
33/545 ... yes
14/546 ... yes
36/547 ... yes
45/549 ... yes
56/550 ... yes
2/551 ... yes
43/552 ... yes
36/553 ... yes
9/554 ... yes
36/555 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.6)
Git version >= 2.24.0 ? ... yes (2.28.0)
Git user has default SSH configuration? ... yes
Active users: ... 19
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 6.x - 7.x? ... skipped (elasticsearch is disabled)

Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

No idea.

Edited by Marc Bihlmaier